00000100:00100000:1.0:1788845010.336788:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641305856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5749us (6118us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845010.336814:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 242 00000100:00100000:1.0:1788845010.341374:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641306112 00000100:00100000:1.0:1788845010.341384:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641306112 00000100:00100000:1.0:1788845010.341449:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 243 00000100:00100000:1.0:1788845010.341460:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641306112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845010.341480:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845010.341498:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000022f65c2a/0xb2187301287ab361 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6f19 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845010.341565:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660129280:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845010.341655:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009281540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845010.347021:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660129280:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845010.347046:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000022f65c2a/0xb2187301287ab361 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f19 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.347064:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000022f65c2a/0xb2187301287ab361 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f19 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.347077:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0347d9400, rc 0) 00000100:00100000:1.0:1788845010.347104:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641306112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5646us (6030us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845010.347121:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 243 00000100:00100000:1.0:1788845010.351281:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.360377:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660129536:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845010.360443:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7e00/0xb2187301287ab36f lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f20 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.360467:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845010.360470:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845010.360474:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7e00/0xb2187301287ab36f lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f20 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.360485:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f025f7e00, rc 0) 00000100:00100000:1.0:1788845010.360552:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641306240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8813us (9284us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845010.360560:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 435 00000100:00100000:1.0:1788845010.367076:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.397624:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.417202:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.433048:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641306880 00000100:00100000:1.0:1788845010.433058:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641306880 00002000:00100000:1.0:1788845010.433116:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d880 x1875739641306880/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:116/0 lens 488/0 e 0 to 0 dl 1788845021 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845010.433127:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d880 with x1875739641306880 00000100:00100000:1.0:1788845010.433169:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 212 00000100:00100000:1.0:1788845010.433179:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641306880:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845010.433618:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641306880:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 438us (690us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845010.433626:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 212 00002000:00100000:1.0:1788845010.433633:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d880 x1875739641306880/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:116/0 lens 488/4536 e 0 to 0 dl 1788845021 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845010.433655:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d880 with x1875739641306880 00000100:00100000:1.0:1788845010.453282:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.467879:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641307136 00000100:00100000:1.0:1788845010.467930:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641307136 00002000:00100000:1.0:1788845010.467981:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358fb80 x1875739641307136/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:116/0 lens 488/0 e 0 to 0 dl 1788845021 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845010.467994:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358fb80 with x1875739641307136 00000100:00100000:1.0:1788845010.468032:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 214 00000100:00100000:1.0:1788845010.468043:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641307136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845010.468646:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641307136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 601us (1092us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845010.468655:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 214 00002000:00100000:1.0:1788845010.468663:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358fb80 x1875739641307136/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:116/0 lens 488/4536 e 0 to 0 dl 1788845021 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845010.468671:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358fb80 with x1875739641307136 00010000:00010000:1.0:1788845010.514546:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845010.550737:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641307776 00000100:00100000:1.0:1788845010.550822:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641307776 00000100:00100000:1.0:1788845010.593535:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641308032 00000100:00100000:1.0:1788845010.593562:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641308032 00000100:00100000:1.0:1788845010.593789:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 248 00000100:00100000:1.0:1788845010.593802:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641308032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845010.593832:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845010.593860:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000023c32cac/0xb2187301287ab399 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6f4a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.593961:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000024e03643/0xb2187301287ab2ea lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6ea9 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.593971:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 9) 00010000:00010000:1.0:1788845010.593977:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 9) 00010000:00010000:1.0:1788845010.593983:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000023c32cac/0xb2187301287ab399 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1597440->1605631] (req 1597440->1605631) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f4a expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.594035:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000024e03643/0xb2187301287ab2ea lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6ea9 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.594057:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000024e03643/0xb2187301287ab2ea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6ea9 expref: 5 pid: 6684 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845010.594074:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beabb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660130304:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845010.594219:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009281c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845010.598768:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beabb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660130304:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845010.598824:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000023c32cac/0xb2187301287ab399 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1597440->1605631] (req 1597440->1605631) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f4a expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.598855:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0347d9c00, rc 0) 00000100:00100000:1.0:1788845010.598969:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641308032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5169us (10109us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845010.599001:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 248 00000100:00100000:1.0:1788845010.616354:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.616492:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.708064:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.737210:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660130688:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845010.737271:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.737300:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845010.737303:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845010.737309:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.737323:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0a00, rc 0) 00000100:00100000:1.0:1788845010.739156:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641308672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 30611us (31098us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845010.739171:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 437 00000100:00100000:1.0:1788845010.742185:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 160 00000100:00100000:1.0:1788845010.742219:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641308800:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845010.742257:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845010.742303:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7e00/0xb2187301287ab36f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f20 expref: 11 pid: 5839 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.742329:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7e00/0xb2187301287ab36f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f20 expref: 11 pid: 5839 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.742380:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7e00/0xb2187301287ab36f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f20 expref: 11 pid: 5839 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.742400:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845010.742408:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.742421:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.742431:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.744240:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845010.744269:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660130816:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845010.744403:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009281c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845010.755157:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660130816:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845010.755200:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845010.755283:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641308800:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 13081us (17431us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845010.755297:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 160 00000100:00100000:1.0:1788845010.805696:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.821692:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660131072:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845010.821761:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1020049 00010000:00010000:1.0:1788845010.821791:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f6600/0xb2187301287ab3b5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f66 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.821818:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845010.821821:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845010.821828:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f6600/0xb2187301287ab3b5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f66 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.821849:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f025f6600, rc 0) 00000100:00100000:1.0:1788845010.822673:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641309312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15856us (16985us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845010.822686:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 439 00000100:00100000:1.0:1788845010.839544:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.843623:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641309568 00000100:00100000:1.0:1788845010.843637:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641309568 00002000:00100000:1.0:1788845010.843705:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dc700 x1875739641309568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:116/0 lens 504/0 e 0 to 0 dl 1788845021 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845010.843718:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dc700 with x1875739641309568 00000100:00100000:1.0:1788845010.843762:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 219 00000100:00100000:1.0:1788845010.843773:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641309568:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845010.886153:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641309568:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 42363us (42659us total) trans 0 rc 901120/901120 00000100:00100000:1.0:1788845010.886172:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 219 00002000:00100000:1.0:1788845010.886192:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dc700 x1875739641309568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:116/0 lens 504/440 e 0 to 0 dl 1788845021 ref 1 fl Complete:/600/0 rc 901120/901120 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845010.886224:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dc700 with x1875739641309568 00000100:00100000:1.0:1788845010.912361:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.946698:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845010.947114:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000029bffb14/0xb2187301287ab3d1 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f7b expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845010.947176:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000d0f681b/0xb2187301287ab3a0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f51 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845010.947198:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000d0f681b/0xb2187301287ab3a0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f51 expref: 7 pid: 6682 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845010.947217:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660131584:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845010.947333:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000023c32cac/0xb2187301287ab399 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1605631) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f4a expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845010.947344:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000023c32cac/0xb2187301287ab399 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1605631) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f4a expref: 5 pid: 7983 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845010.947353:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660131712:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845010.947373:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4480 going to sleep for 11 seconds 00000100:00100000:3.0:1788845010.952118:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660131712:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845010.966975:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660131584:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845010.967037:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000029bffb14/0xb2187301287ab3d1 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f7b expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845010.967068:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1200, rc 0) 00000100:00100000:3.0:1788845010.967285:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641310336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 20302us (20596us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845010.967314:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 250 00000100:00100000:1.0:1788845010.968899:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845010.969157:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845010.971626:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641310464 00000100:00100000:3.0:1788845010.971647:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641310464 00000100:00100000:3.0:1788845010.980012:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660131840:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845010.980043:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845010.988620:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641310848 00000100:00100000:1.0:1788845010.988633:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641310848 00000100:00100000:1.0:1788845010.988771:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 443 00000100:00100000:1.0:1788845010.988783:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641310848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845010.988818:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845010.988846:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1c00/0xb2187301287ab3d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6f82 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.989001:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab3df lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.989020:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab3df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.989034:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab3df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845010.989070:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1676400 00000004:00200000:1.0:1788845010.989075:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845010.989109:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab3df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.989142:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1c00/0xb2187301287ab3d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f82 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.989206:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab3df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f82 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845010.989216:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845010.989220:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845010.989236:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02ee1600, rc 0) 00000100:00100000:1.0:1788845010.989634:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641310848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 852us (1353us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845010.989643:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 443 00000100:00100000:1.0:1788845011.000312:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.000781:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641310976 00000100:00100000:3.0:1788845011.000796:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641310976 00000100:00100000:3.0:1788845011.000931:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 251 00000100:00100000:3.0:1788845011.000945:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035445180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641310976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.000972:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.000998:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b15e354/0xb2187301287ab3e6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6f89 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.001048:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000029bffb14/0xb2187301287ab3d1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f7b expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.001058:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(0 11) 00010000:00010000:3.0:1788845011.001064:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:3.0:1788845011.001069:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b15e354/0xb2187301287ab3e6 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1474560->1544191] (req 1474560->1544191) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f89 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.008128:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660131968:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845011.008175:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b15e354/0xb2187301287ab3e6 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1474560->1544191] (req 1474560->1544191) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f89 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.008203:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1400, rc 0) 00000100:00100000:1.0:1788845011.008292:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035445180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641310976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7350us (7996us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845011.008303:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 251 00000100:00080000:3.0:1788845011.009079:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845011.026211:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.031999:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c58380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660132096:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845011.032036:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845011.032117:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035444700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641311232:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5420us (5924us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845011.032156:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 164 00000100:00100000:1.0:1788845011.038578:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641311488 00000100:00100000:1.0:1788845011.038591:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641311488 00002000:00100000:1.0:1788845011.038656:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1478656->1482751) req@ffff949f02483800 x1875739641311488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.038669:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02483800 with x1875739641311488 ext(1478656->1482751) 00002000:00010000:1.0:1788845011.038694:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02483800 00000100:00100000:1.0:1788845011.038736:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 113 00000100:00100000:1.0:1788845011.038754:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641311488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.039399:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641311488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 651us (1012us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.039409:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 113 00002000:00100000:1.0:1788845011.039419:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1478656->1482751) req@ffff949f02483800 x1875739641311488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.039429:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02483800 with x1875739641311488 ext(1478656->1482751) 00002000:00010000:1.0:1788845011.039433:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02483800 00000100:00100000:1.0:1788845011.044581:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641311616 00000100:00100000:1.0:1788845011.044595:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641311616 00002000:00100000:1.0:1788845011.044666:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1482752->1486847) req@ffff949f01c58380 x1875739641311616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.044678:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c58380 with x1875739641311616 ext(1482752->1486847) 00002000:00010000:1.0:1788845011.044685:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c58380 00000100:00100000:1.0:1788845011.044766:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 114 00000100:00100000:1.0:1788845011.044779:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641311616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.045151:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641311616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 372us (784us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.045160:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 114 00002000:00100000:1.0:1788845011.045167:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1482752->1486847) req@ffff949f01c58380 x1875739641311616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.045177:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c58380 with x1875739641311616 ext(1482752->1486847) 00002000:00010000:1.0:1788845011.045188:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c58380 00000100:00100000:1.0:1788845011.054326:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641311872 00000100:00100000:1.0:1788845011.054340:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641311872 00002000:00100000:1.0:1788845011.054421:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1490944->1495039) req@ffff949f02482d80 x1875739641311872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.054435:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02482d80 with x1875739641311872 ext(1490944->1495039) 00002000:00010000:1.0:1788845011.054442:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02482d80 00000100:00100000:1.0:1788845011.054474:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 116 00000100:00100000:1.0:1788845011.054486:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641311872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.054851:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641311872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 365us (730us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.054860:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 116 00002000:00100000:1.0:1788845011.054869:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1490944->1495039) req@ffff949f02482d80 x1875739641311872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.054879:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02482d80 with x1875739641311872 ext(1490944->1495039) 00002000:00010000:1.0:1788845011.054920:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02482d80 00000100:00100000:3.0:1788845011.060080:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.060406:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641312000 00000100:00100000:1.0:1788845011.060421:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641312000 00002000:00100000:1.0:1788845011.060489:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1495040->1499135) req@ffff949f0383e300 x1875739641312000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.060503:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383e300 with x1875739641312000 ext(1495040->1499135) 00002000:00010000:1.0:1788845011.060510:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383e300 00000100:00100000:1.0:1788845011.060550:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 117 00000100:00100000:1.0:1788845011.060561:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641312000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.060964:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641312000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 402us (898us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.060974:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 117 00002000:00100000:1.0:1788845011.060982:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1495040->1499135) req@ffff949f0383e300 x1875739641312000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.060991:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383e300 with x1875739641312000 ext(1495040->1499135) 00002000:00010000:1.0:1788845011.060994:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383e300 00000100:00100000:3.0:1788845011.069310:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.074255:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.074397:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641312384 00000100:00100000:1.0:1788845011.074412:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641312384 00002000:00100000:1.0:1788845011.074485:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1507328->1511423) req@ffff949f0383c700 x1875739641312384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.074498:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383c700 with x1875739641312384 ext(1507328->1511423) 00002000:00010000:1.0:1788845011.074506:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383c700 00000100:00100000:1.0:1788845011.074550:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 120 00000100:00100000:1.0:1788845011.074562:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641312384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.074933:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641312384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 371us (692us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.074941:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 120 00002000:00100000:1.0:1788845011.074949:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1507328->1511423) req@ffff949f0383c700 x1875739641312384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.074959:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383c700 with x1875739641312384 ext(1507328->1511423) 00002000:00010000:1.0:1788845011.074963:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383c700 00000100:00100000:3.0:1788845011.078664:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.083477:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.088036:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.088162:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641312768 00000100:00100000:1.0:1788845011.088192:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641312768 00002000:00100000:1.0:1788845011.088268:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1519616->1523711) req@ffff94a036082300 x1875739641312768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.088281:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036082300 with x1875739641312768 ext(1519616->1523711) 00002000:00010000:1.0:1788845011.088289:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036082300 00000100:00100000:1.0:1788845011.088332:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 123 00000100:00100000:1.0:1788845011.088344:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641312768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.088700:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641312768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 356us (677us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.088709:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 123 00002000:00100000:1.0:1788845011.088717:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1519616->1523711) req@ffff94a036082300 x1875739641312768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.088726:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036082300 with x1875739641312768 ext(1519616->1523711) 00002000:00010000:1.0:1788845011.088729:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036082300 00000100:00100000:3.0:1788845011.092829:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.097131:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.101999:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.102343:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641313152 00000100:00100000:1.0:1788845011.102356:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641313152 00002000:00100000:1.0:1788845011.102427:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1531904->1535999) req@ffff94a036080a80 x1875739641313152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.102440:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036080a80 with x1875739641313152 ext(1531904->1535999) 00002000:00010000:1.0:1788845011.102448:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036080a80 00000100:00100000:1.0:1788845011.102492:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 126 00000100:00100000:1.0:1788845011.102503:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641313152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.102829:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641313152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 326us (846us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.102859:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 126 00002000:00100000:1.0:1788845011.102867:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1531904->1535999) req@ffff94a036080a80 x1875739641313152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.102877:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036080a80 with x1875739641313152 ext(1531904->1535999) 00002000:00010000:1.0:1788845011.102911:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036080a80 00000100:00100000:3.0:1788845011.107510:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.113139:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.118807:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.128008:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.128142:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641313664 00000100:00100000:1.0:1788845011.128157:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641313664 00000100:00100000:1.0:1788845011.128247:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 445 00000100:00100000:1.0:1788845011.128260:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641313664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.128288:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.128317:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1400/0xb2187301287ab3fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6f97 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.128370:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.128441:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035446300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660132224:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.128568:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009281c00 going to sleep for 11 seconds 00000004:00010000:1.0:1788845011.132428:0:11036:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 901120 -> 1020049 00000004:00010000:1.0:1788845011.132443:0:11036:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845010 -> 1788845011 00000004:00010000:1.0:1788845011.132445:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845008 -> 1788845010 00000004:00010000:1.0:1788845011.132447:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845010 -> 1788845011 00000004:00010000:1.0:1788845011.132459:0:11036:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 390 -> 648 00000100:00100000:1.0:1788845011.132498:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035446300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660132224:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845011.132532:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1020049 00010000:00010000:1.0:1788845011.132579:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1400/0xb2187301287ab3fb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f97 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.132620:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845011.132624:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845011.132631:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1400/0xb2187301287ab3fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f97 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.132650:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02ee1400, rc 0) 00000100:00100000:1.0:1788845011.132732:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641313664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4473us (4739us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845011.132751:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 445 00000100:00100000:3.0:1788845011.139363:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641313792 00000100:00100000:3.0:1788845011.139376:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641313792 00000100:00100000:3.0:1788845011.139470:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 446 00000100:00100000:3.0:1788845011.139483:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641313792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.139512:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.139538:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0a00/0xb2187301287ab402 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6f9e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.139589:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.139645:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036082d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660132352:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845011.140052:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.149918:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641313920 00000100:00100000:3.0:1788845011.149931:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641313920 00000100:00100000:3.0:1788845011.149999:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 252 00000100:00100000:3.0:1788845011.150012:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641313920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.150037:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.150061:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b3fc064/0xb2187301287ab409 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6fa5 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.150101:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:3.0:1788845011.150106:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:3.0:1788845011.150119:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b3fc064/0xb2187301287ab409 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fa5 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.150138:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b3fc064/0xb2187301287ab409 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fa5 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.150156:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0c00, rc 0) 00000100:00100000:3.0:1788845011.150248:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641313920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 238us (644us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845011.150255:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 252 00000100:00100000:3.0:1788845011.159974:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641314176 00000100:00100000:3.0:1788845011.159991:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641314176 00000100:00100000:3.0:1788845011.160114:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 447 00000100:00100000:3.0:1788845011.160126:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641314176:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845011.160212:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845011.160302:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab3df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f82 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.160324:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.160332:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845011.160628:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.160647:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab3f4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f90 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.160658:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.160699:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab3f4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f90 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.160722:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab3f4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f90 expref: 9 pid: 5840 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.160863:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660132480:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845011.161051:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.167087:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660132480:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845011.168417:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.168428:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.168449:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845011.168691:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967513 00000004:00010000:3.0:1788845011.168810:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1020049 -> 901120 00000004:00010000:3.0:1788845011.168815:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845011 -> 1788845011 00000004:00010000:3.0:1788845011.168818:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845010 -> 1788845008 00000004:00010000:3.0:1788845011.168820:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845011 -> 1788845011 00000004:00010000:3.0:1788845011.168822:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 390 00000004:00010000:3.0:1788845011.168831:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000095dcc018 reply state 0000000014bcd5b4 transno 4294967514 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.168841:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845011.168990:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641314176:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8863us (9461us total) trans 4294967514 rc 0/0 00000100:00100000:3.0:1788845011.168998:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 447 00000020:00080000:3.0:1788845011.169003:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845011.176428:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.176458:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.176468:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.176657:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01cb1c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845011.176664:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.176704:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.176730:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.176734:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845011.176909:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.176919:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab410 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.184405:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.193381:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641314560 00000100:00100000:1.0:1788845011.193394:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641314560 00010000:00010000:1.0:1788845011.193456:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002b15e354/0xb2187301287ab3e6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1474560->1544191) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f89 expref: 6 pid: 6685 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.193510:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641314688 00000100:00100000:1.0:1788845011.193514:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641314688 00010000:00010000:1.0:1788845011.193529:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b3fc064/0xb2187301287ab409 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fa5 expref: 11 pid: 7983 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.193551:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 166 00000100:00100000:1.0:1788845011.193575:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082fe450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641314560:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845011.193595:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845011.193627:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002b15e354/0xb2187301287ab3e6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1474560->1544191) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f89 expref: 6 pid: 6685 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.193667:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002b15e354/0xb2187301287ab3e6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1474560->1544191) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f89 expref: 6 pid: 6685 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.193706:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845011.193792:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082fe450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641314560:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 231us (752us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845011.193801:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 166 00000100:00100000:1.0:1788845011.193863:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 167 00000100:00100000:1.0:1788845011.193870:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641314688:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845011.193878:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845011.193921:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b3fc064/0xb2187301287ab409 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fa5 expref: 11 pid: 7983 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.193941:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b3fc064/0xb2187301287ab409 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fa5 expref: 11 pid: 7983 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.193955:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:1.0:1788845011.193961:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:1.0:1788845011.193970:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000b56418f/0xb2187301287ab417 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fac expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.193981:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000b56418f/0xb2187301287ab417 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fac expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.194195:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000b56418f/0xb2187301287ab417 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fac expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.194213:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035447100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660132864:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845011.194291:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009281540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.207751:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.207988:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035447100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660132864:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845011.207996:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:1.0:1788845011.208038:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845011.208128:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641314688:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 14256us (15039us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845011.208139:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 167 00000100:00100000:1.0:1788845011.208190:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641314944 00000100:00100000:1.0:1788845011.208198:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641314944 00002000:00010000:1.0:1788845011.208236:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845011.208240:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036081c00 with x1875739641314944 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845011.208251:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036081c00. 00000100:00100000:1.0:1788845011.208282:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 129 00000100:00100000:1.0:1788845011.208291:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641314944:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845011.208591:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2097152 -> 0 00002000:00010000:1.0:1788845011.208596:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845011 -> 1788845011 00002000:00010000:1.0:1788845011.208599:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845010 -> 1788845010 00002000:00010000:1.0:1788845011.208601:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845011 -> 1788845011 00002000:00010000:1.0:1788845011.208603:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 259 -> 259 00000100:00100000:1.0:1788845011.208625:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641314944:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 335us (636us total) trans 4294967347 rc 0/0 00000100:00100000:1.0:1788845011.208645:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 129 00002000:00010000:1.0:1788845011.208648:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845011.208651:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036081c00 with x1875739641314944 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845011.208654:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036081c00. 00000100:00100000:1.0:1788845011.210540:0:5854:0:(service.c:3462:ptlrpc_start_thread()) mdt_io[0] started 3 min 3 max 116 00000100:00100000:1.0:1788845011.210565:0:5854:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_io00_003' 00000100:00100000:3.0:1788845011.216722:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.227120:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.227191:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.237664:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660133120:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845011.237697:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845011.237720:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7a00/0xb2187301287ab42c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fba expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.237743:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845011.237746:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845011.237751:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7a00/0xb2187301287ab42c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fba expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.237767:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f025f7a00, rc 0) 00000100:00100000:1.0:1788845011.240225:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035447100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641315200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12864us (13099us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845011.240238:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 449 00000100:00100000:1.0:1788845011.244646:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.244685:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.253083:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660133376:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845011.253100:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845011.253112:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f6800/0xb2187301287ab441 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fc8 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.253126:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.253128:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845011.253131:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f6800/0xb2187301287ab441 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fc8 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.253139:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f025f6800, rc 0) 00000100:00100000:3.0:1788845011.253159:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035444700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641315456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8208us (8524us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845011.253164:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 450 00000100:00100000:1.0:1788845011.258384:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641315840 00000100:00100000:1.0:1788845011.258394:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641315840 00000100:00100000:1.0:1788845011.258457:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 256 00000100:00100000:1.0:1788845011.258481:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641315840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.258510:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.258529:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000036999f59/0xb2187301287ab448 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6fdd expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.258585:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660133504:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.258670:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009397e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.262868:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641315712 00000100:00100000:3.0:1788845011.262909:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641315712 00000100:00100000:3.0:1788845011.281433:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641315968 00000100:00100000:3.0:1788845011.281444:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641315968 00000100:00100000:3.0:1788845011.281511:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 452 00000100:00100000:3.0:1788845011.281521:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641315968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.281548:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.281573:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab456 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6fe4 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.281614:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.281660:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660133760:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845011.281763:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845011.282328:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641316096 00000100:00100000:1.0:1788845011.282337:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641316096 00000100:00100000:1.0:1788845011.282380:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 257 00000100:00100000:1.0:1788845011.282388:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641316096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.282399:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.282410:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6adcfd9/0xb2187301287ab45d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6feb expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.282449:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660133888:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.282836:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009397e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845011.290390:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660133888:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845011.290436:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6adcfd9/0xb2187301287ab45d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6feb expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.290463:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6adcfd9/0xb2187301287ab45d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6feb expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.290477:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02ee1600, rc 0) 00000100:00100000:1.0:1788845011.290552:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641316096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8164us (9298us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845011.290561:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 257 00000100:00100000:3.0:1788845011.291340:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660133760:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845011.291358:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845011.291374:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab456 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fe4 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.291390:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.291393:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845011.291397:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab456 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fe4 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.291409:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0200, rc 0) 00000100:00100000:3.0:1788845011.291493:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641315968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9973us (10185us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845011.291513:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 452 00000100:00100000:1.0:1788845011.309055:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641316224 00000100:00100000:1.0:1788845011.309840:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641316224 00000100:00100000:1.0:1788845011.320630:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641316352 00000100:00100000:1.0:1788845011.320639:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641316352 00000100:00100000:1.0:1788845011.320702:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 453 00000100:00100000:1.0:1788845011.320712:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641316352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.320734:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.320767:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab464 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a6ff2 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.320810:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.320851:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660134016:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.320967:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009397e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.332396:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660134144:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845011.332437:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c838273f/0xb2187301287ab46b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6ff9 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.332464:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c838273f/0xb2187301287ab46b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6ff9 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.332478:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d16a00, rc 0) 00000100:00100000:3.0:1788845011.332542:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641316480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10900us (12008us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845011.332551:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 258 00000100:00100000:1.0:1788845011.332670:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660134016:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845011.332685:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845011.332697:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab464 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6ff2 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.332711:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845011.332714:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845011.332718:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1600/0xb2187301287ab464 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6ff2 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.333020:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02ee1600, rc 0) 00000100:00100000:1.0:1788845011.333049:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641316352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12338us (12580us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845011.334018:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 453 00000100:00100000:1.0:1788845011.346745:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641316608 00000100:00100000:1.0:1788845011.346854:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641316608 00000100:00100000:1.0:1788845011.347283:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7aed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660134272:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845011.347319:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004158ab3e/0xb2187301287ab472 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7007 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.347343:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004158ab3e/0xb2187301287ab472 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7007 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.347355:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d16600, rc 0) 00000100:00100000:1.0:1788845011.347408:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641316736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3456us (3795us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845011.347416:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 259 00000100:00100000:1.0:1788845011.353510:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 454 00000100:00100000:1.0:1788845011.353527:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641316608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.353551:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.353575:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1c00/0xb2187301287ab479 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7000 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.353617:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.353679:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660134400:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.353796:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009397e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.357169:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660134400:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845011.357207:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845011.357238:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1c00/0xb2187301287ab479 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7000 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.357271:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.357275:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845011.357280:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee1c00/0xb2187301287ab479 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7000 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.357297:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02ee1c00, rc 0) 00000100:00100000:3.0:1788845011.357366:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641316608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3843us (14209us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845011.357389:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 454 00000100:00100000:1.0:1788845011.360667:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641316864 00000100:00100000:1.0:1788845011.360677:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641316864 00000100:00100000:1.0:1788845011.360830:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:1.0:1788845011.360841:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641316864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.360866:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.360928:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0600/0xb2187301287ab480 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a700e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.361009:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0c00/0xb2187301287ab487 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.361023:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0c00/0xb2187301287ab487 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.361033:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0c00/0xb2187301287ab487 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845011.361066:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 890784 00000004:00200000:1.0:1788845011.361069:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845011.361086:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0c00/0xb2187301287ab487 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.361110:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0600/0xb2187301287ab480 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a700e expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.361127:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0c00/0xb2187301287ab487 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a700e expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.361149:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845011.361152:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845011.361155:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02ee0c00, rc 0) 00000100:00100000:1.0:1788845011.361209:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641316864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 370us (653us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845011.361215:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:3.0:1788845011.369054:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.369243:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641317120 00000100:00100000:1.0:1788845011.369252:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641317120 00000100:00100000:1.0:1788845011.369309:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 260 00000100:00100000:1.0:1788845011.369317:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641317120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.369335:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.369351:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6adcfd9/0xb2187301287ab495 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a701c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.369404:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660134656:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.369472:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009397e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845011.372545:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660134656:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845011.372593:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6adcfd9/0xb2187301287ab495 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a701c expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.372620:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6adcfd9/0xb2187301287ab495 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a701c expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.372634:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02ee1600, rc 0) 00000100:00100000:1.0:1788845011.372717:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641317120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3398us (3669us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845011.372727:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 260 00000100:00100000:1.0:1788845011.377470:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641317248 00000100:00100000:1.0:1788845011.377480:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641317248 00000100:00100000:1.0:1788845011.377545:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 457 00000100:00100000:1.0:1788845011.377555:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641317248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.377577:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.377598:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0600/0xb2187301287ab49c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7023 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.377638:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.377682:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660134784:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.377785:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009397e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845011.382986:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660134784:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845011.383018:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845011.383037:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0600/0xb2187301287ab49c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7023 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.383069:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845011.383072:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845011.383077:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0600/0xb2187301287ab49c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7023 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.383093:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02ee0600, rc 0) 00000100:00100000:1.0:1788845011.383178:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641317248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5624us (5777us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845011.383187:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 457 00000100:00100000:3.0:1788845011.389931:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641317632 00000100:00100000:3.0:1788845011.389942:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641317632 00000100:00100000:3.0:1788845011.390005:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 458 00000100:00100000:3.0:1788845011.390016:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641317632:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845011.390068:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845011.390106:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02ee0c00/0xb2187301287ab487 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a700e expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.390125:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.390129:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845011.390511:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab4aa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.390528:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab425 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fb3 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.390536:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab4aa lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.390568:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab425 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fb3 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.390587:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab425 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fb3 expref: 9 pid: 5840 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.390601:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660135040:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845011.390753:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845011.395603:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845011.398431:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab425 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6fb3 expref: 9 pid: 5840 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.398458:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660135040:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845011.398483:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab4aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.405247:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.405996:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641317888 00000100:00100000:3.0:1788845011.406005:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641317888 00002000:00010000:3.0:1788845011.406042:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1926194->18446744073709551615) 00002000:00100000:3.0:1788845011.406047:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dd500 with x1875739641317888 ext(1926194->18446744073709551615) 00002000:00010000:3.0:1788845011.406154:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dd500. 00000100:00100000:3.0:1788845011.406230:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 130 00000100:00100000:3.0:1788845011.406240:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641317888:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845011.406440:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1926194 00002000:00010000:3.0:1788845011.406443:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845011 -> 1788845011 00002000:00010000:3.0:1788845011.406446:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845010 -> 1788845010 00002000:00010000:3.0:1788845011.406448:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845011 -> 1788845011 00002000:00010000:3.0:1788845011.406450:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 259 -> 259 00000100:00100000:3.0:1788845011.406511:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641317888:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 271us (1271us total) trans 4294967348 rc 0/0 00000100:00100000:3.0:1788845011.406518:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 130 00002000:00010000:3.0:1788845011.406521:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1926194->18446744073709551615) 00002000:00100000:3.0:1788845011.406524:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dd500 with x1875739641317888 ext(1926194->18446744073709551615) 00002000:00010000:3.0:1788845011.406528:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dd500. 00000100:00100000:3.0:1788845011.409727:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641318016 00000100:00100000:3.0:1788845011.409859:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641318016 00000100:00100000:3.0:1788845011.410065:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 459 00000100:00100000:3.0:1788845011.410077:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641318016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.410112:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.410139:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4b1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7031 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.411012:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0000/0xb2187301287ab4b8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.411030:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0000/0xb2187301287ab4b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.411040:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0000/0xb2187301287ab4b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845011.411077:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1926194 00000004:00200000:3.0:1788845011.411081:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845011.411104:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0000/0xb2187301287ab4b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.411132:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4b1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7031 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.411150:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0000/0xb2187301287ab4b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7031 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.411158:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.411160:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845011.411177:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0000, rc 0) 00000100:00100000:3.0:1788845011.411283:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641318016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1208us (1660us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845011.411289:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 459 00000100:00100000:3.0:1788845011.417258:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641318272 00000100:00100000:3.0:1788845011.417263:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641318272 00000100:00100000:3.0:1788845011.417298:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 262 00000100:00100000:3.0:1788845011.417304:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641318272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.417313:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.417324:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000023d38bc8/0xb2187301287ab4c6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a703f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.417357:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660135296:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845011.417386:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.424698:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660135296:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845011.425063:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000023d38bc8/0xb2187301287ab4c6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a703f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.425100:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000023d38bc8/0xb2187301287ab4c6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a703f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.425115:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1000, rc 0) 00000100:00100000:3.0:1788845011.429665:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641318400 00000100:00100000:3.0:1788845011.429675:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641318400 00000100:00100000:3.0:1788845011.429744:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 461 00000100:00100000:3.0:1788845011.429763:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641318400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.429799:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.429823:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7046 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.429853:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.429864:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4cd lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7046 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.429920:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.429939:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 22 pid: 11036 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.429953:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660135424:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845011.430593:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.436234:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660135424:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845011.436284:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4cd lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7046 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.436304:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.436307:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845011.436312:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4cd lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7046 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.436342:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1c00, rc 0) 00000100:00100000:3.0:1788845011.436392:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641318400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6632us (6809us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845011.436402:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 461 00000100:00100000:3.0:1788845011.437627:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641318528 00000100:00100000:3.0:1788845011.437637:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641318528 00010000:00010000:3.0:1788845011.437673:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 23 pid: 11036 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.437701:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 169 00000100:00100000:3.0:1788845011.437710:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641318528:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845011.437723:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845011.437956:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 23 pid: 11036 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.437975:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 23 pid: 11036 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.437997:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ab3a7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a6f58 expref: 23 pid: 11036 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.438011:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.438017:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7046 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.438067:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4cd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7046 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.438077:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7046 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.448233:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641318656 00000100:00100000:1.0:1788845011.448245:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641318656 00002000:00100000:1.0:1788845011.448411:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684700 x1875739641318656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.448421:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684700 with x1875739641318656 00000100:00100000:1.0:1788845011.448508:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 222 00000100:00100000:1.0:1788845011.448517:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641318656:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.513932:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641318656:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 65404us (66888us total) trans 0 rc 892928/892928 00000100:00100000:1.0:1788845011.513947:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 222 00002000:00100000:1.0:1788845011.513961:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684700 x1875739641318656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/440 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 892928/892928 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.513972:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684700 with x1875739641318656 00000100:00100000:3.0:1788845011.519984:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.525447:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641318912 00000100:00100000:3.0:1788845011.525457:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641318912 00000100:00100000:3.0:1788845011.525522:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 463 00000100:00100000:3.0:1788845011.525533:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641318912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.525560:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.525563:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845011.525610:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7046 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.525630:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.525635:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845011.525647:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab4db lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7054 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.525670:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17200/0xb2187301287ab4d4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a704d expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.525680:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab4db lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7054 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.525731:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17200/0xb2187301287ab4d4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a704d expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.525761:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17200/0xb2187301287ab4d4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a704d expref: 22 pid: 5840 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.525776:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036082d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660135680:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845011.525980:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845011.531632:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.535287:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036082d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660135680:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845011.535326:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab4db lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7054 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.535347:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.535350:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845011.535355:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab4db lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7054 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.535367:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1600, rc 0) 00000100:00100000:3.0:1788845011.535432:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641318912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9900us (10081us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845011.535442:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 463 00000100:00100000:1.0:1788845011.548768:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.548859:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641319168 00000100:00100000:3.0:1788845011.548870:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641319168 00000100:00100000:3.0:1788845011.548959:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 223 00000100:00100000:3.0:1788845011.548970:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641319168:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845011.549282:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890784 -> 890784 00000004:00010000:3.0:1788845011.549286:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845011 -> 1788845011 00000004:00010000:3.0:1788845011.549289:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845010 -> 1788845010 00000004:00010000:3.0:1788845011.549291:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845011 -> 1788845011 00000004:00010000:3.0:1788845011.549292:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:3.0:1788845011.549364:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641319168:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 396us (607us total) trans 4294967519 rc 0/0 00000100:00100000:3.0:1788845011.549373:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 223 00000100:00100000:3.0:1788845011.552758:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.565955:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.578945:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.579033:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641319680 00000100:00100000:3.0:1788845011.579044:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641319680 00000100:00100000:3.0:1788845011.579110:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 465 00000100:00100000:3.0:1788845011.579121:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641319680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.579144:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.579165:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7070 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.579197:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab4db lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7054 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.579207:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4fe lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7070 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.579253:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab4db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7054 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.579298:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab4db lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7054 expref: 11 pid: 5839 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.579312:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660136192:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845011.579391:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.587484:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.587593:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660136192:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845011.587626:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4fe lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7070 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.587670:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845011.587672:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845011.587677:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab4fe lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7070 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.587687:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1c00, rc 0) 00000100:00100000:1.0:1788845011.587761:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641319680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8641us (8826us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845011.587769:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 465 00000100:00100000:1.0:1788845011.609656:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.609704:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.609838:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641320064 00000100:00100000:3.0:1788845011.609849:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641320064 00002000:00100000:3.0:1788845011.609942:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1929215) req@ffff94a008919880 x1875739641320064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845011.609954:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a008919880 with x1875739641320064 ext(1048576->1929215) 00002000:00010000:3.0:1788845011.609959:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a008919880 00000100:00100000:3.0:1788845011.609992:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 131 00000100:00100000:3.0:1788845011.610003:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641320064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.668129:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660136448 00000100:00100000:1.0:1788845011.668137:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660136448 00000100:00100000:1.0:1788845011.668189:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 72 00000100:00100000:1.0:1788845011.668198:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660136448:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788845011.668246:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 49348608 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788845011.668427:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660136448:12345-0@lo:13:osp-pre-0-0.0 Request processed in 229us (508us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845011.668434:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 72 00000100:00100000:1.0:1788845011.691610:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.691671:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.707968:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.732682:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641320704 00000100:00100000:1.0:1788845011.732695:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641320704 00010000:00010000:1.0:1788845011.732762:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16400/0xb2187301287ab505 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7077 expref: 12 pid: 5840 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.733011:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 172 00000100:00100000:1.0:1788845011.733022:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641320704:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845011.733040:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845011.733072:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16400/0xb2187301287ab505 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7077 expref: 12 pid: 5840 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.733102:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16400/0xb2187301287ab505 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7077 expref: 12 pid: 5840 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.733131:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845011.733137:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.733150:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.733159:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.733217:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.733234:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660136960:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845011.733503:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009397e40 going to sleep for 11 seconds 00010000:00010000:3.0:1788845011.733755:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16400/0xb2187301287ab505 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7077 expref: 12 pid: 5840 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.733774:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660136832:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845011.733803:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.733815:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.733818:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845011.733830:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17c00, rc 0) 00000100:00100000:3.0:1788845011.733863:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641320576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7806us (8345us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845011.733872:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 467 00000100:00100000:1.0:1788845011.750672:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660136960:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845011.750727:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845011.750965:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641320704:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 17933us (18379us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845011.750981:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 172 00000100:00080000:3.0:1788845011.758082:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845011.759124:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641320832 00000100:00100000:1.0:1788845011.759139:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641320832 00000100:00100000:1.0:1788845011.759225:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 273 00000100:00100000:1.0:1788845011.759236:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2367:x1875739641320832:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845011.759339:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2367:x1875739641320832:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 104us (3045us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845011.759347:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 273 00010000:00010000:1.0:1788845011.759392:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845011.759714:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641321088 00000100:00100000:3.0:1788845011.759719:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641321088 00002000:00100000:3.0:1788845011.759774:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f774e00 x1875739641321088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845011.759786:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f774e00 with x1875739641321088 00000100:00100000:3.0:1788845011.759813:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 225 00000100:00100000:3.0:1788845011.759820:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641321088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845011.760127:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641321088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 306us (574us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845011.760135:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 225 00002000:00100000:3.0:1788845011.760142:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f774e00 x1875739641321088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845011.760151:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f774e00 with x1875739641321088 00000100:00100000:3.0:1788845011.766331:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641321216 00000100:00100000:3.0:1788845011.766344:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641321216 00002000:00100000:3.0:1788845011.766411:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454a80 x1875739641321216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845011.766424:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454a80 with x1875739641321216 00000100:00100000:3.0:1788845011.766471:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 226 00000100:00100000:3.0:1788845011.766483:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641321216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.773459:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641321472 00000100:00100000:1.0:1788845011.773472:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641321472 00000100:00100000:1.0:1788845011.773551:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 268 00000100:00100000:1.0:1788845011.773563:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641321472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.773589:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.773616:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ab521 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a709a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.773701:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660137088:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.774111:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0016719c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.777780:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660137088:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845011.777839:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ab521 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a709a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.777872:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ab521 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a709a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.777919:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:3.0:1788845011.777994:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641321472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4433us (4627us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845011.778004:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 268 00000100:00100000:1.0:1788845011.785420:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641321600 00000100:00100000:1.0:1788845011.785432:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641321600 00000100:00100000:1.0:1788845011.785509:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 469 00000100:00100000:1.0:1788845011.785522:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641321600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.785571:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.785604:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287ab52f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a70a1 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.785655:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.785713:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008919c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660137344:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845011.786693:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0016719c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845011.789643:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008919c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660137344:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845011.789688:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845011.789716:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287ab52f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70a1 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.789762:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845011.789766:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845011.789772:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287ab52f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70a1 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.789793:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0400, rc 0) 00000100:00100000:1.0:1788845011.789874:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641321600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4353us (4696us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845011.789901:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 469 00000100:00100000:1.0:1788845011.792302:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.796064:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03455c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660137472:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845011.796182:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004a4c1113/0xb2187301287ab536 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70a8 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.796228:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a4c1113/0xb2187301287ab536 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70a8 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.796246:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d16800, rc 0) 00000100:00100000:3.0:1788845011.796535:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641321728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10524us (10736us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845011.796545:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 269 00000100:00100000:1.0:1788845011.800684:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641321856 00000100:00100000:1.0:1788845011.800697:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641321856 00000100:00100000:1.0:1788845011.801031:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 470 00000100:00100000:1.0:1788845011.801061:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641321856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845011.801158:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845011.801187:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ab53d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a70af expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.801235:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.801247:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ab53d lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70af expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.801368:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.801391:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 22 pid: 5839 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845011.801407:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008919c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660137600:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845011.801559:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0016719c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845011.813602:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008919c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660137600:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845011.813678:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ab53d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70af expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.813706:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.813709:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845011.813714:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ab53d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70af expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.813738:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0200, rc 0) 00000100:00100000:3.0:1788845011.813826:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641321856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12766us (13252us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845011.813837:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 470 00000004:00010000:1.0:1788845011.827824:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890784 -> 890784 00000004:00010000:1.0:1788845011.827835:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845011 -> 1788845011 00000004:00010000:1.0:1788845011.827837:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845011 -> 1788845010 00000004:00010000:1.0:1788845011.827840:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845011 -> 1788845011 00000004:00010000:1.0:1788845011.827842:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:1.0:1788845011.828402:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641321984:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 13040us (14927us total) trans 4294967520 rc 0/0 00000100:00100000:1.0:1788845011.828428:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 11 00002000:00100000:1.0:1788845011.828443:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741f80 x1875739641321984/t4294967520(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:117/0 lens 488/448 e 0 to 0 dl 1788845022 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.828452:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741f80 with x1875739641321984 00010000:00010000:1.0:1788845011.828462:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 25 pid: 5839 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845011.828484:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 25 pid: 5839 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845011.828492:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a035741f80 00000100:00100000:3.0:1788845011.837294:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641322112 00000100:00100000:3.0:1788845011.837314:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641322112 00010000:00010000:3.0:1788845011.837385:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 25 pid: 5839 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.837438:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 173 00000100:00100000:3.0:1788845011.837451:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082fc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641322112:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845011.837471:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845011.837505:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 25 pid: 5839 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.837528:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 25 pid: 5839 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.837556:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab51a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a708c expref: 25 pid: 5839 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.837595:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.837604:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ab53d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70af expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.837618:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ab53d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70af expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.837628:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ab53d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70af expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.837680:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ab53d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70af expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845011.837697:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660137728:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845011.883863:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641322368 00000100:00100000:3.0:1788845011.883910:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641322368 00000100:00100000:3.0:1788845011.883992:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:3.0:1788845011.884001:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:2367:x1875739641322368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845011.884029:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845011.884054:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab544 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a70b6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.884147:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab544 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70b6 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845011.884190:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845011.884202:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab544 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70b6 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845011.884228:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845011.884231:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845011.884235:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1000, rc 0) 00000100:00100000:3.0:1788845011.884312:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641322368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 311us (561us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845011.884318:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:1.0:1788845011.890559:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.913073:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.917146:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.923025:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.929151:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.935683:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.940485:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.948522:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.954664:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845011.962695:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.973585:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641323776 00000100:00100000:1.0:1788845011.973600:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641323776 00002000:00100000:1.0:1788845011.976603:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff94a035b7e300 x1875739641323776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.976619:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7e300 with x1875739641323776 ext(1245184->1249279) 00002000:00010000:1.0:1788845011.976627:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7e300 00000100:00100000:1.0:1788845011.976679:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 136 00000100:00100000:1.0:1788845011.976698:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641323776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.976987:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641323776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 297us (14300us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.976995:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 136 00002000:00100000:1.0:1788845011.977017:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff94a035b7e300 x1875739641323776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.977025:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7e300 with x1875739641323776 ext(1245184->1249279) 00002000:00010000:1.0:1788845011.977030:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7e300 00000100:00100000:3.0:1788845011.991371:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641324032 00000100:00100000:3.0:1788845011.991389:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641324032 00002000:00100000:3.0:1788845011.991464:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1257471) req@ffff94a035743480 x1875739641324032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845011.991477:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035743480 with x1875739641324032 ext(1253376->1257471) 00002000:00010000:3.0:1788845011.991485:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035743480 00000100:00100000:3.0:1788845011.991526:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 138 00000100:00100000:3.0:1788845011.991536:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641324032:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845011.992018:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641324032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 482us (737us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845011.992025:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 138 00002000:00100000:3.0:1788845011.992031:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1257471) req@ffff94a035743480 x1875739641324032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845011.992037:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035743480 with x1875739641324032 ext(1253376->1257471) 00002000:00010000:3.0:1788845011.992040:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035743480 00000100:00100000:3.0:1788845011.995111:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845011.996025:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641324160 00000100:00100000:1.0:1788845011.996041:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641324160 00002000:00100000:1.0:1788845011.996102:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1257472->1261567) req@ffff94a035b7dc00 x1875739641324160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/0 e 0 to 0 dl 1788845022 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.996115:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7dc00 with x1875739641324160 ext(1257472->1261567) 00002000:00010000:1.0:1788845011.996121:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7dc00 00000100:00100000:1.0:1788845011.996164:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 139 00000100:00100000:1.0:1788845011.996174:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641324160:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845011.996566:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641324160:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 392us (1466us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845011.996575:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 139 00002000:00100000:1.0:1788845011.996582:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1257472->1261567) req@ffff94a035b7dc00 x1875739641324160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:117/0 lens 488/4536 e 0 to 0 dl 1788845022 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845011.996590:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7dc00 with x1875739641324160 ext(1257472->1261567) 00002000:00010000:1.0:1788845011.996593:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7dc00 00000100:00100000:3.0:1788845012.000988:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641324288 00000100:00100000:3.0:1788845012.001005:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641324288 00000100:00100000:3.0:1788845012.001100:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 271 00000100:00100000:3.0:1788845012.001114:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035446d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2368:x1875739641324288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.001141:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.001167:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c5f7a601/0xb2187301287ab559 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a70cb expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.001200:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:3.0:1788845012.001214:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:3.0:1788845012.001227:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c5f7a601/0xb2187301287ab559 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70cb expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.001248:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c5f7a601/0xb2187301287ab559 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70cb expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.001265:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0e00, rc 0) 00000100:00100000:3.0:1788845012.001375:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641324288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 264us (543us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845012.001382:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 271 00000100:00100000:1.0:1788845012.005631:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641324416 00000100:00100000:1.0:1788845012.005644:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641324416 00000100:00100000:1.0:1788845012.005724:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 473 00000100:00100000:1.0:1788845012.005972:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641324416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.006001:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.006047:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ab560 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a70d2 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.006168:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788845012.011261:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845010 -> 1788845011 00000100:00100000:1.0:1788845012.011310:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035740a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660138368:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845012.011347:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845012.011373:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ab560 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70d2 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.011398:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.011401:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845012.011407:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ab560 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70d2 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.011426:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845012.011542:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641324416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5806us (6049us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.011552:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 473 00000100:00100000:3.0:1788845012.015663:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641324544 00000100:00100000:3.0:1788845012.015675:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641324544 00000100:00100000:3.0:1788845012.016027:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 474 00000100:00100000:3.0:1788845012.016041:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641324544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.016071:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.016170:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab567 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a70d9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.016226:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.016292:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660138496:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.016455:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.026100:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660138496:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845012.026144:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845012.026175:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab567 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70d9 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.026215:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845012.026219:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845012.026226:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab567 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70d9 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.026246:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845012.026362:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641324544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10323us (10817us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.026373:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 474 00000100:00100000:3.0:1788845012.031959:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641324800 00000100:00100000:3.0:1788845012.031973:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641324800 00000100:00100000:3.0:1788845012.032053:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 475 00000100:00100000:3.0:1788845012.032065:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641324800:12345-192.168.201.29@tcp:36:fsx.0 00000100:00100000:1.0:1788845012.042186:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845012.042391:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641324928 00000100:00100000:3.0:1788845012.042403:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641324928 00010000:00010000:3.0:1788845012.042468:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16600/0xb2187301287ab4e9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a705b expref: 11 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.042519:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 176 00000100:00100000:3.0:1788845012.042531:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641324928:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845012.042550:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845012.042558:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16600/0xb2187301287ab4e9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a705b expref: 11 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.042579:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16600/0xb2187301287ab4e9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a705b expref: 11 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.042610:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845012.042617:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.042629:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.042663:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.042687:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845012.043020:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641324928:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 490us (848us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845012.043029:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 176 00000100:00100000:1.0:1788845012.050531:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845012.051667:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.051678:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.051685:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.051691:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f025f7800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845012.051696:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.051704:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.051735:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845012.051844:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845012.051850:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.051857:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f7800/0xb2187301287ab56e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.055489:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.055527:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641325312 00000100:00100000:1.0:1788845012.055539:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641325312 00010000:00010000:1.0:1788845012.055589:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006eb641c2/0xb2187301287ab552 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1228800->1261567) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70c4 expref: 6 pid: 7983 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.055665:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 177 00000100:00100000:1.0:1788845012.055677:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641325312:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845012.055695:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845012.055726:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006eb641c2/0xb2187301287ab552 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1228800->1261567) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70c4 expref: 6 pid: 7983 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.055874:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006eb641c2/0xb2187301287ab552 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1228800->1261567) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70c4 expref: 6 pid: 7983 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.055948:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845012.056015:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641325312:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 339us (541us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845012.056024:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 177 00000100:00100000:1.0:1788845012.056073:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.056100:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641325184 00000100:00100000:1.0:1788845012.056104:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641325184 00010000:00010000:1.0:1788845012.056121:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c5f7a601/0xb2187301287ab559 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70cb expref: 13 pid: 6682 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.056144:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 178 00000100:00100000:1.0:1788845012.056151:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641325184:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845012.056160:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845012.056169:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c5f7a601/0xb2187301287ab559 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70cb expref: 13 pid: 6682 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.056201:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c5f7a601/0xb2187301287ab559 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70cb expref: 13 pid: 6682 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.056227:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:1.0:1788845012.056232:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:1.0:1788845012.056241:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab575 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e0 expref: 13 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.056254:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab575 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e0 expref: 13 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.056298:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab575 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e0 expref: 13 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.056317:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660139008:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845012.056382:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147d80 going to sleep for 11 seconds 00010000:00010000:3.0:1788845012.057637:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c5f7a601/0xb2187301287ab559 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70cb expref: 13 pid: 6682 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.057664:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660138752:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845012.057682:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006eb641c2/0xb2187301287ab552 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1228800->1261567) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70c4 expref: 5 pid: 7983 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.057694:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660138880:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845012.057919:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab575 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e0 expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.057935:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d16c00, rc 0) 00000100:00100000:3.0:1788845012.057996:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:14891:x1875739641325056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7127us (7480us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845012.058006:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 272 00000100:00100000:1.0:1788845012.065025:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.074228:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.080145:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641325696 00000100:00100000:1.0:1788845012.080159:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641325696 00000100:00100000:1.0:1788845012.080282:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 477 00000100:00100000:1.0:1788845012.080295:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641325696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.080324:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.080352:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a70ee expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.080410:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.080422:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.080481:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.080503:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 26 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.080521:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660139136:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845012.080635:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.085122:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.085171:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641325824 00000100:00100000:1.0:1788845012.085184:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641325824 00002000:00100000:1.0:1788845012.085259:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035328000 x1875739641325824/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:118/0 lens 16872/0 e 0 to 0 dl 1788845023 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845012.085273:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328000 with x1875739641325824 00010000:00010000:1.0:1788845012.085283:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 27 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.085305:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 27 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845012.085333:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a035328000 00000100:00100000:1.0:1788845012.085371:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 12 00000100:00100000:1.0:1788845012.085382:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641325824:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845012.085821:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890784 -> 890784 00000004:00010000:1.0:1788845012.085826:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845012 -> 1788845012 00000004:00010000:1.0:1788845012.085828:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845010 -> 1788845010 00000004:00010000:1.0:1788845012.085831:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845012 -> 1788845012 00000004:00010000:1.0:1788845012.085834:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:1.0:1788845012.085927:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641325824:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 545us (821us total) trans 4294967523 rc 0/0 00000100:00100000:1.0:1788845012.085937:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 12 00002000:00100000:1.0:1788845012.085947:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035328000 x1875739641325824/t4294967523(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:118/0 lens 16872/448 e 0 to 0 dl 1788845023 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845012.085955:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328000 with x1875739641325824 00010000:00010000:1.0:1788845012.085962:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 29 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.085976:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 29 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845012.085983:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a035328000 00000100:00100000:3.0:1788845012.094306:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.094477:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641325952 00000100:00100000:1.0:1788845012.094491:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641325952 00010000:00010000:1.0:1788845012.094545:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 29 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.094624:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 179 00000100:00100000:1.0:1788845012.094636:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641325952:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845012.094656:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845012.094693:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 29 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.094714:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 29 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.094754:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16e00/0xb2187301287ab54b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70bd expref: 29 pid: 5840 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.094775:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.094782:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.094796:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.094806:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.094854:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.094940:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660139264:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845012.095048:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.100559:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660139264:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845012.100592:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845012.100665:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641325952:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6030us (6375us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845012.100680:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 179 00000100:00100000:1.0:1788845012.107642:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 230 00000100:00100000:1.0:1788845012.107665:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641326080:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845012.108015:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890784 -> 890784 00000004:00010000:1.0:1788845012.108021:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845012 -> 1788845012 00000004:00010000:1.0:1788845012.108023:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845010 -> 1788845010 00000004:00010000:1.0:1788845012.108035:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845012 -> 1788845012 00000004:00010000:1.0:1788845012.108038:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:1.0:1788845012.108144:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641326080:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 486us (7784us total) trans 4294967524 rc 0/0 00000100:00100000:1.0:1788845012.108152:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 230 00000100:00100000:3.0:1788845012.113755:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641326208 00000100:00100000:3.0:1788845012.113767:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641326208 00000100:00100000:3.0:1788845012.113829:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:3.0:1788845012.113839:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641326208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.113863:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.113904:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab591 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a70f5 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.114006:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0e00/0xb2187301287ab598 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.114020:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0e00/0xb2187301287ab598 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.114031:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0e00/0xb2187301287ab598 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845012.114054:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1057601 00000004:00200000:3.0:1788845012.114058:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845012.114074:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0e00/0xb2187301287ab598 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.114099:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab591 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70f5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.114115:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0e00/0xb2187301287ab598 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70f5 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.114122:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845012.114125:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845012.114128:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0e00, rc 0) 00000100:00100000:3.0:1788845012.114185:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641326208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 348us (1223us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845012.114192:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:3.0:1788845012.118946:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845012.130076:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.132972:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 274 00000100:00100000:1.0:1788845012.132983:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641326464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.133001:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.133019:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ab5a6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7103 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.133096:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660139520:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.133184:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.139068:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660139520:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845012.139114:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ab5a6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7103 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.139140:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ab5a6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7103 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.139154:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845012.139246:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641326464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6265us (9179us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.139255:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 274 00000100:00100000:1.0:1788845012.192014:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845012.192433:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641326848 00000100:00100000:3.0:1788845012.192443:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641326848 00000100:00100000:3.0:1788845012.192498:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 275 00000100:00100000:3.0:1788845012.192506:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641326848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.192524:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.192540:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab5bb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7118 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.192593:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660139904:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.192798:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4e40 going to sleep for 11 seconds 00002000:00010000:3.0:1788845012.199622:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845011 -> 1788845012 00000100:00100000:3.0:1788845012.199666:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660139904:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845012.199703:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab5bb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7118 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.199728:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab5bb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7118 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.199740:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845012.199863:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641326848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7356us (7859us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.199872:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 275 00000100:00100000:3.0:1788845012.207607:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641326976 00000100:00100000:3.0:1788845012.207617:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641326976 00000100:00100000:3.0:1788845012.207743:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 276 00000100:00100000:3.0:1788845012.207755:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641326976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.207789:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.207809:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab5c2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a711f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.207870:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660140032:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.208045:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.212084:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660140032:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845012.212126:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab5c2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a711f expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.212153:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab5c2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a711f expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.212167:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845012.212240:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641326976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4487us (4753us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.212249:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 276 00000100:00100000:1.0:1788845012.244788:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845012.271686:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641327232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 26570us (26891us total) trans 0 rc 892928/892928 00000100:00100000:3.0:1788845012.271703:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 231 00002000:00100000:3.0:1788845012.271727:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035329880 x1875739641327232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:118/0 lens 488/440 e 0 to 0 dl 1788845023 ref 1 fl Complete:/600/0 rc 892928/892928 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845012.271956:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035329880 with x1875739641327232 00000100:00100000:1.0:1788845012.281223:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641327360 00000100:00100000:1.0:1788845012.281233:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641327360 00000100:00100000:1.0:1788845012.281294:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 481 00000100:00100000:1.0:1788845012.281304:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641327360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.281328:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.281350:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ab5d0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a712d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.281386:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.281436:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660140288:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.281512:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.285599:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660140288:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845012.285637:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845012.285664:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ab5d0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a712d expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.285691:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.285695:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845012.285701:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ab5d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a712d expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.285731:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845012.286084:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641327360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4771us (4987us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.286094:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 481 00000100:00100000:3.0:1788845012.295675:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641327488 00000100:00100000:3.0:1788845012.295737:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641327488 00000100:00100000:3.0:1788845012.295815:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 482 00000100:00100000:3.0:1788845012.295829:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641327488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.295857:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.295934:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab5d7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7134 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.295985:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.296035:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660140416:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.296151:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.300475:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660140416:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845012.300516:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845012.300542:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab5d7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7134 expref: 29 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.300570:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845012.300573:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845012.300580:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab5d7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7134 expref: 29 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.300600:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845012.300680:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641327488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4854us (5478us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.300690:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 482 00000100:00100000:1.0:1788845012.327730:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641327872 00000100:00100000:1.0:1788845012.328137:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641327872 00000100:00100000:1.0:1788845012.328269:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 485 00000100:00100000:1.0:1788845012.328283:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641327872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.328311:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.328340:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ab5ec lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7149 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.328404:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.328485:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660140800:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.328592:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.332926:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660140800:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845012.332964:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845012.332988:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ab5ec lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7149 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.333013:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.333016:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845012.333030:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ab5ec lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7149 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.333048:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845012.333122:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641327872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4841us (5570us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.333131:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 485 00000100:00100000:3.0:1788845012.339699:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641328000 00000100:00100000:3.0:1788845012.339845:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641328000 00000100:00100000:3.0:1788845012.340005:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:3.0:1788845012.340018:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641328000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.340591:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.340623:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab5f3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7150 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.340676:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.341249:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660140928:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.346759:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660140928:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845012.346801:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845012.346830:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab5f3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7150 expref: 29 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.346861:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845012.346864:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845012.346869:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab5f3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7150 expref: 29 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.346925:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845012.347010:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641328000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6994us (7688us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.347029:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:3.0:1788845012.354236:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.354357:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641328128 00000100:00100000:1.0:1788845012.354385:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641328128 00000100:00100000:1.0:1788845012.354475:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 278 00000100:00100000:1.0:1788845012.354489:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641328128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.354517:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.354541:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ab5fa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7157 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.354638:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660141056:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.354777:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.365876:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660141056:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845012.366222:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ab5fa lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7157 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.366257:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ab5fa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7157 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.366275:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845012.366407:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641328128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11920us (12185us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.366417:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 278 00000100:00100000:3.0:1788845012.372854:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641328256 00000100:00100000:3.0:1788845012.373215:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641328256 00000100:00100000:3.0:1788845012.373363:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 279 00000100:00100000:3.0:1788845012.373377:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641328256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.373403:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.373428:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab601 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a715e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.373510:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660141184:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.373665:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.382593:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660141184:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845012.382652:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab601 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a715e expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.382684:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab601 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a715e expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.382699:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845012.383266:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641328256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9887us (10635us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.383278:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 279 00000100:00100000:1.0:1788845012.415191:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641328640 00000100:00100000:1.0:1788845012.415213:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641328640 00000100:00100000:1.0:1788845012.415291:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 489 00000100:00100000:1.0:1788845012.415303:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641328640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.415332:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.415359:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab616 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7173 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.415408:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.415462:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660141568:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.415576:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.420157:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660141568:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845012.420210:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845012.420240:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab616 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7173 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.420383:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.420388:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845012.420395:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab616 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7173 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.420416:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845012.420501:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641328640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5198us (5534us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.420511:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 489 00000100:00100000:3.0:1788845012.426872:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641328768 00000100:00100000:3.0:1788845012.426921:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641328768 00000100:00100000:3.0:1788845012.426989:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 490 00000100:00100000:3.0:1788845012.427001:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641328768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.427041:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.427098:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab61d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a717a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.427150:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.427212:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660141696:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.448182:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641329024 00000100:00100000:3.0:1788845012.448205:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641329024 00000100:00100000:3.0:1788845012.448334:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 280 00000100:00100000:3.0:1788845012.448348:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641329024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.448374:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.448398:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000023d38bc8/0xb2187301287ab624 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7181 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.448490:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660141824:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.448602:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.452375:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660141824:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845012.452444:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000023d38bc8/0xb2187301287ab624 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7181 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.452480:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000023d38bc8/0xb2187301287ab624 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7181 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.452499:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1000, rc 0) 00000100:00100000:3.0:1788845012.452588:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641329024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4241us (4671us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.452614:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 280 00000100:00100000:1.0:1788845012.474361:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.497159:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.502580:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359af480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660142336:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845012.502624:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845012.502651:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab640 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a719d expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.502678:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.502681:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845012.502688:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab640 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a719d expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.502708:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023af000, rc 0) 00000100:00100000:1.0:1788845012.503088:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641329536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5634us (5944us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.503119:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 493 00000100:00100000:1.0:1788845012.540257:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.556551:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359adc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660142592:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845012.556600:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c5ce56c9/0xb2187301287ab64e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71ab expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.556624:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c5ce56c9/0xb2187301287ab64e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71ab expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.556639:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023af200, rc 0) 00000100:00100000:1.0:1788845012.556854:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641329792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11888us (16605us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.556864:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 282 00000100:00100000:1.0:1788845012.565743:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.568392:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359adc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660142720:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845012.568434:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000066bcaa34/0xb2187301287ab655 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71b2 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.568461:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000066bcaa34/0xb2187301287ab655 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71b2 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.568477:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023afc00, rc 0) 00000100:00100000:1.0:1788845012.568543:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641329920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2534us (2810us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.568567:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 283 00000100:00100000:1.0:1788845012.573709:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845012.573857:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641330048 00000100:00100000:3.0:1788845012.573868:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641330048 00000100:00100000:3.0:1788845012.573956:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 284 00000100:00100000:3.0:1788845012.573966:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641330048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.573984:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.574003:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000023d38bc8/0xb2187301287ab65c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a71b9 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.574060:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660142848:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.574159:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.607263:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641330176 00000100:00100000:3.0:1788845012.607274:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641330176 00000100:00100000:3.0:1788845012.607344:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 285 00000100:00100000:3.0:1788845012.607354:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641330176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.607373:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.607392:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab663 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a71c0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.607449:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660142976:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.607534:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.616627:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660142976:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845012.616685:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab663 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71c0 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.616826:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab663 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71c0 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.616853:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0000, rc 0) 00000100:00100000:3.0:1788845012.617009:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641330176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9654us (10006us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.617018:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 285 00000100:00100000:1.0:1788845012.627684:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641330304 00000100:00100000:1.0:1788845012.627694:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641330304 00000100:00100000:1.0:1788845012.627758:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 286 00000100:00100000:1.0:1788845012.627767:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641330304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.627785:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.627803:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ab66a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a71c7 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.627859:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360eb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660143104:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.627966:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.636565:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360eb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660143104:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845012.636607:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ab66a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71c7 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.636630:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ab66a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71c7 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.636657:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:1.0:1788845012.636722:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641330304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8955us (9149us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.636732:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 286 00000100:00100000:1.0:1788845012.649211:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641330432 00000100:00100000:1.0:1788845012.649233:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641330432 00000100:00100000:1.0:1788845012.649294:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 287 00000100:00100000:1.0:1788845012.649305:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641330432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.649325:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.649345:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ab671 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a71ce expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.649415:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360eb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660143232:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.649516:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.652253:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360eb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660143232:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845012.652294:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ab671 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71ce expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.652318:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ab671 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71ce expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.652331:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:1.0:1788845012.652392:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641330432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3088us (3328us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.652402:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 287 00000100:00100000:1.0:1788845012.714568:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641330688 00000100:00100000:1.0:1788845012.714580:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641330688 00000100:00100000:1.0:1788845012.714682:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 496 00000100:00100000:1.0:1788845012.714694:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641330688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.714732:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.714766:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ab67f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a71dc expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.714807:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.714847:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360eb100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660143488:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.714957:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.743937:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641330944 00000100:00100000:3.0:1788845012.743947:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641330944 00000100:00100000:3.0:1788845012.744004:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 289 00000100:00100000:3.0:1788845012.744014:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641330944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.744033:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.744052:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab68d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a71ea expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.744110:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660143744:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.744205:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.761844:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660143744:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845012.761928:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab68d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71ea expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.761954:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab68d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71ea expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.761988:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0000, rc 0) 00000100:00100000:3.0:1788845012.762048:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641330944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 18035us (18221us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.762056:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 289 00000100:00100000:1.0:1788845012.773754:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641331072 00000100:00100000:1.0:1788845012.773765:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641331072 00000100:00100000:1.0:1788845012.773832:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 497 00000100:00100000:1.0:1788845012.773842:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641331072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.773864:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.773914:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a71f1 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.773947:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.773956:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.773993:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.774010:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 13 pid: 11036 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.774042:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660143872:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845012.774127:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0031470c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.790872:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641331200 00000100:00100000:3.0:1788845012.790916:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641331200 00010000:00010000:3.0:1788845012.790958:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 14 pid: 11036 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.790999:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 180 00000100:00100000:3.0:1788845012.791007:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082fec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641331200:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845012.791023:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845012.791049:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 14 pid: 11036 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.791065:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 14 pid: 11036 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.791097:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845012.791103:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.791114:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.791141:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.791177:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.791190:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660144000:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845012.791287:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4c00 going to sleep for 11 seconds 00010000:00010000:1.0:1788845012.791873:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ab58a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70ee expref: 14 pid: 11036 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.791912:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660143872:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845012.791937:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.791948:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.791951:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845012.791954:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845012.791996:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:14891:x1875739641331072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 18156us (18828us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845012.792005:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 497 00000100:00100000:3.0:1788845012.804172:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660144000:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845012.804229:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845012.804287:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082fec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641331200:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 13279us (13614us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845012.804298:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 180 00000100:00100000:1.0:1788845012.804852:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641331328 00000100:00100000:1.0:1788845012.804861:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641331328 00002000:00100000:1.0:1788845012.804940:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010483100 x1875739641331328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:118/0 lens 488/0 e 0 to 0 dl 1788845023 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845012.804951:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010483100 with x1875739641331328 00000100:00100000:1.0:1788845012.810993:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 233 00000100:00100000:1.0:1788845012.811022:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641331328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845012.811998:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641331328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 985us (7372us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845012.812019:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 233 00002000:00100000:1.0:1788845012.812030:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010483100 x1875739641331328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:118/0 lens 488/4536 e 0 to 0 dl 1788845023 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845012.812039:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010483100 with x1875739641331328 00000100:00100000:1.0:1788845012.831653:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641331456 00000100:00100000:1.0:1788845012.831665:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641331456 00002000:00100000:1.0:1788845012.831738:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010483800 x1875739641331456/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:118/0 lens 488/0 e 0 to 0 dl 1788845023 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845012.831750:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010483800 with x1875739641331456 00000100:00100000:1.0:1788845012.831796:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 234 00000100:00100000:1.0:1788845012.831808:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+29:2367:x1875739641331456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845012.832459:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+29:2367:x1875739641331456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 651us (1047us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845012.832469:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 234 00002000:00100000:1.0:1788845012.832479:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010483800 x1875739641331456/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:118/0 lens 488/4536 e 0 to 0 dl 1788845023 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845012.832486:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010483800 with x1875739641331456 00000100:00100000:1.0:1788845012.846424:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641331712 00000100:00100000:1.0:1788845012.846436:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641331712 00000100:00100000:1.0:1788845012.846515:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 290 00000100:00100000:1.0:1788845012.846526:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641331712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845012.846547:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845012.846566:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ab69b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a71ff expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.846648:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660144128:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845012.846754:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.849434:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660144128:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845012.849471:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ab69b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71ff expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.849494:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ab69b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a71ff expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.849508:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845012.849570:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641331712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3046us (3246us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845012.849579:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 290 00000100:00100000:3.0:1788845012.885020:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641331968 00000100:00100000:3.0:1788845012.885025:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641331968 00000100:00100000:3.0:1788845012.885076:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 291 00000100:00100000:3.0:1788845012.885085:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641331968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845012.885094:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845012.885106:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab6b0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a720d expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.885141:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660144512:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845012.885173:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845012.889250:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660144512:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845012.889286:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab6b0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a720d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.889305:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab6b0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a720d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.889315:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0000, rc 0) 00000100:00100000:3.0:1788845012.889339:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641331968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4256us (5029us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845012.889346:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 291 00000100:00100000:3.0:1788845012.895978:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.896085:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641332224 00000100:00100000:1.0:1788845012.896094:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641332224 00000100:00100000:1.0:1788845012.896155:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 500 00000100:00100000:1.0:1788845012.896164:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641332224:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845012.896231:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845012.896276:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0e00/0xb2187301287ab598 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70f5 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.896297:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.896302:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845012.896528:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.896544:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f6200/0xb2187301287ab583 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e7 expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.896553:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.896586:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f6200/0xb2187301287ab583 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e7 expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.896605:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f6200/0xb2187301287ab583 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e7 expref: 28 pid: 5840 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.896620:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660144640:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845012.896712:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147f00 going to sleep for 11 seconds 00010000:00010000:1.0:1788845012.902235:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f025f6200/0xb2187301287ab583 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e7 expref: 28 pid: 5840 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.902275:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660144640:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845012.902301:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.902312:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.902317:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845012.902471:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967526 00000004:00010000:1.0:1788845012.902528:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890784 -> 890784 00000004:00010000:1.0:1788845012.902532:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845012 -> 1788845012 00000004:00010000:1.0:1788845012.902534:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845012 -> 1788845010 00000004:00010000:1.0:1788845012.902536:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845012 -> 1788845012 00000004:00010000:1.0:1788845012.902539:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000004:00010000:1.0:1788845012.902546:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001918a2ed reply state 000000000c5ad66e transno 4294967527 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.902555:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845012.902666:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641332224:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6501us (6695us total) trans 4294967527 rc 0/0 00000100:00100000:1.0:1788845012.902674:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 500 00000020:00080000:1.0:1788845012.902680:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845012.907877:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.907932:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.907939:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.907947:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845012.907952:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.907960:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.907978:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845012.907982:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845012.908001:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.908009:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ab6b7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845012.909474:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845012.915930:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641332608 00000100:00100000:3.0:1788845012.915940:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641332608 00002000:00100000:3.0:1788845012.915990:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c4a80 x1875739641332608/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:118/0 lens 488/0 e 0 to 0 dl 1788845023 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845012.916002:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c4a80 with x1875739641332608 00010000:00010000:3.0:1788845012.916010:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 28 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845012.916029:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 28 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845012.916037:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a03f6c4a80 00000100:00100000:3.0:1788845012.916063:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 13 00000100:00100000:3.0:1788845012.916072:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641332608:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845012.927435:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:1.0:1788845012.930656:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ab694 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a71f1 expref: 30 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.930685:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660144768:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845012.930736:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.930755:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845012.930758:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845012.930763:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023ae200, rc 0) 00000100:00100000:1.0:1788845012.930835:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641332480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 20098us (21367us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845012.930865:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 501 00000100:00100000:3.0:1788845012.934781:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845012.946911:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641332992 00000100:00100000:1.0:1788845012.946923:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641332992 00010000:00010000:1.0:1788845012.946969:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab575 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e0 expref: 12 pid: 6684 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.947018:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 183 00000100:00100000:1.0:1788845012.947029:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:2369:x1875739641332992:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845012.947043:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845012.947079:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab575 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e0 expref: 12 pid: 6684 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.947098:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab575 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a70e0 expref: 12 pid: 6684 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.947126:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:1.0:1788845012.947131:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:1.0:1788845012.947140:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000486618c1/0xb2187301287ab6c5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a721b expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.947151:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000486618c1/0xb2187301287ab6c5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a721b expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845012.947200:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000486618c1/0xb2187301287ab6c5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a721b expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845012.947236:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660145152:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845012.947667:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845012.962645:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660145152:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845012.962680:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845012.962751:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641332992:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 15722us (16595us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845012.962763:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 183 00000100:00100000:3.0:1788845012.974362:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.004406:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641333504 00000100:00100000:3.0:1788845013.004412:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641333504 00000100:00100000:3.0:1788845013.004468:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 293 00000100:00100000:3.0:1788845013.004477:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641333504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.004502:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.004516:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab6e1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7230 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.004556:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660145408:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.004605:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4b40 going to sleep for 11 seconds 00002000:00010000:3.0:1788845013.020503:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845010 -> 1788845012 00000100:00100000:1.0:1788845013.025895:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660145408:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845013.025951:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab6e1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7230 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.025979:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000efbb3204/0xb2187301287ab6e1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7230 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.025993:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0000, rc 0) 00000100:00100000:1.0:1788845013.026292:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641333504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 21814us (23172us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.026302:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 293 00000100:00100000:1.0:1788845013.038580:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641333760 00000100:00100000:1.0:1788845013.038585:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641333760 00000100:00100000:1.0:1788845013.038626:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 294 00000100:00100000:1.0:1788845013.038635:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2367:x1875739641333760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.038662:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.038675:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab6ef lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a723e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.038721:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660145664:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845013.038750:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.041507:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660145664:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845013.041553:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab6ef lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a723e expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.041579:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab6ef lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a723e expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.041594:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845013.041655:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2367:x1875739641333760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3021us (3634us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.041663:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 294 00000100:00100000:3.0:1788845013.054624:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.054677:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.054785:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641334016 00000100:00100000:1.0:1788845013.054795:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641334016 00000100:00100000:1.0:1788845013.054866:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 295 00000100:00100000:1.0:1788845013.054878:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2367:x1875739641334016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.054934:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.054954:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab6f6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a724c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.055049:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660145792:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845013.055140:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.069417:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360e9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660145792:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845013.069459:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab6f6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a724c expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.069484:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab6f6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a724c expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.069498:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845013.069566:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2367:x1875739641334016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 14688us (14951us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.069575:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 295 00000100:00100000:1.0:1788845013.095879:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.095935:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.095962:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641334272 00000100:00100000:1.0:1788845013.095971:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641334272 00000100:00100000:1.0:1788845013.096027:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 296 00000100:00100000:1.0:1788845013.096039:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641334272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.096059:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.096079:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab704 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a725a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.096139:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ea680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660146048:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845013.096239:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.103357:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ea680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660146048:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845013.103410:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab704 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a725a expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.103438:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ab704 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a725a expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.103450:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845013.118725:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641334272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 22677us (22778us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.118738:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 296 00000100:00100000:3.0:1788845013.130214:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641334400 00000100:00100000:3.0:1788845013.130228:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641334400 00000100:00100000:3.0:1788845013.130298:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 297 00000100:00100000:3.0:1788845013.131297:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:14891:x1875739641334400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.131318:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.131339:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7261 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.131372:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000486618c1/0xb2187301287ab6c5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a721b expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.131381:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:3.0:1788845013.131385:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:3.0:1788845013.131391:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1544192->1609727] (req 1544192->1609727) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.131427:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000486618c1/0xb2187301287ab6c5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a721b expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.131460:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000486618c1/0xb2187301287ab6c5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a721b expref: 6 pid: 6684 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.131477:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660146304:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845013.131569:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845013.136952:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660146304:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845013.136997:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1544192->1609727] (req 1544192->1609727) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.137022:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0800, rc 0) 00000100:00100000:3.0:1788845013.137304:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:14891:x1875739641334400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6994us (8655us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845013.137313:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 297 00000100:00080000:1.0:1788845013.150949:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845013.153326:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.153923:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660146432:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845013.153946:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845013.153998:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641334528:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4861us (5884us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845013.154007:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 184 00000100:00100000:1.0:1788845013.161034:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845013.163616:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab719 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7268 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.163681:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.163727:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660146560:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845013.163830:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.170838:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660146560:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845013.170870:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:1.0:1788845013.171365:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab719 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7268 expref: 31 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.171390:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845013.171394:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845013.171399:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab719 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7268 expref: 31 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.171415:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17000, rc 0) 00000100:00100000:1.0:1788845013.171537:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+30:2367:x1875739641334784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7968us (10511us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.171546:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 507 00000100:00100000:1.0:1788845013.179755:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.190692:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.191199:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641335040 00000100:00100000:1.0:1788845013.191210:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641335040 00000100:00100000:1.0:1788845013.191276:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 509 00000100:00100000:1.0:1788845013.191289:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+30:14891:x1875739641335040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.191312:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.191337:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7276 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.191370:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.191381:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 31 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.191414:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.191433:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 14 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.191446:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360e8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660146816:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845013.191530:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.196118:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.196225:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641335168 00000100:00100000:0.0:1788845013.196235:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641335168 00010000:00010000:0.0:1788845013.196298:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 15 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.196342:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 185 00000100:00100000:0.0:1788845013.196353:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641335168:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845013.196368:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845013.196399:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 15 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.196420:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 15 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.196449:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae200/0xb2187301287ab6be lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7214 expref: 15 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.196467:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.196474:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.196499:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.196510:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.196555:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.196571:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660146944:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845013.196655:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c480 going to sleep for 11 seconds 00000100:00080000:1.0:1788845013.197103:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845013.202778:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660146944:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845013.202810:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845013.202865:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641335168:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6514us (6753us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845013.202878:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 185 00000100:00100000:0.0:1788845013.202952:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.202979:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641335296 00000100:00100000:0.0:1788845013.202986:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641335296 00000100:00100000:0.0:1788845013.203049:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 235 00000100:00100000:0.0:1788845013.203058:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641335296:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845013.203332:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890784 -> 890784 00000004:00010000:0.0:1788845013.203337:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845012 -> 1788845012 00000004:00010000:0.0:1788845013.203340:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845012 -> 1788845012 00000004:00010000:0.0:1788845013.203353:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845012 -> 1788845012 00000004:00010000:0.0:1788845013.203355:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:0.0:1788845013.203393:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641335296:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 336us (440us total) trans 4294967529 rc 0/0 00000100:00100000:0.0:1788845013.203400:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 235 00000100:00100000:3.0:1788845013.209055:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.209102:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641335424 00000100:00100000:3.0:1788845013.209111:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641335424 00000100:00100000:3.0:1788845013.209169:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 510 00000100:00100000:3.0:1788845013.209201:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641335424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.209221:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.209242:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1800/0xb2187301287ab72e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a727d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.209335:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1a00/0xb2187301287ab735 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.209352:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1a00/0xb2187301287ab735 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.209364:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1a00/0xb2187301287ab735 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845013.209394:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1659206 00000004:00200000:3.0:1788845013.209400:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845013.209420:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1a00/0xb2187301287ab735 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.209451:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1800/0xb2187301287ab72e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a727d expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.209472:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1a00/0xb2187301287ab735 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a727d expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.209482:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.209485:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845013.209489:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1a00, rc 0) 00000100:00100000:3.0:1788845013.209550:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641335424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 363us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845013.209567:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 510 00000100:00100000:3.0:1788845013.224745:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641335680 00000100:00100000:3.0:1788845013.224754:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641335680 00000100:00100000:3.0:1788845013.224811:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 298 00000100:00100000:3.0:1788845013.224820:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641335680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.224837:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.224854:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000de05d7ce/0xb2187301287ab743 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a728b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.224917:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660147200:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.224983:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4b40 going to sleep for 11 seconds 00000004:00010000:3.0:1788845013.235401:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845012 -> 1788845013 00000100:00100000:3.0:1788845013.235442:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660147072:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845013.235472:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845013.235497:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab73c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7284 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.235522:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.235525:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845013.235530:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab73c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7284 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.235544:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023af600, rc 0) 00000100:00100000:3.0:1788845013.235771:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641335552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12414us (12525us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845013.235781:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 511 00002000:00010000:1.0:1788845013.236033:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845010 -> 1788845012 00002000:00010000:1.0:1788845013.236038:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845012 -> 1788845013 00000100:00100000:1.0:1788845013.236056:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660147200:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845013.236080:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000de05d7ce/0xb2187301287ab743 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a728b expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.236099:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000de05d7ce/0xb2187301287ab743 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a728b expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.236110:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1e00, rc 0) 00000100:00100000:1.0:1788845013.236138:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641335680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11320us (12995us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.236145:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 298 00000100:00100000:3.0:1788845013.239995:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.240036:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.240110:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641335936 00000100:00100000:1.0:1788845013.240121:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641335936 00000100:00100000:3.0:1788845013.240180:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641335808 00000100:00100000:3.0:1788845013.240194:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641335808 00000100:00100000:3.0:1788845013.240330:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 512 00000100:00100000:3.0:1788845013.240343:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641335808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.240365:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.240385:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab74a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7292 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.240421:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 31 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.240460:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660147328:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.240536:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4480 going to sleep for 11 seconds 00000100:00100000:3.0:1788845013.246280:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660147328:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845013.246310:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845013.246331:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab74a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7292 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.246353:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.246356:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845013.246361:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab74a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7292 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.246376:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1c00, rc 0) 00000100:00100000:3.0:1788845013.246431:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641335808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6090us (6391us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845013.246448:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 512 00000100:00100000:1.0:1788845013.246509:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 299 00000100:00100000:1.0:1788845013.246521:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641335936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.246538:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.246555:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ab751 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7299 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.246615:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360eb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660147456:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845013.246672:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.257119:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360eb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660147456:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845013.257175:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ab751 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7299 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.257220:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ab751 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7299 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.257233:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845013.257296:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641335936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10777us (17310us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.257306:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 299 00000100:00100000:0.0:1788845013.268413:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.287440:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.287482:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641336192 00000100:00100000:3.0:1788845013.287492:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641336192 00010000:00010000:3.0:1788845013.287539:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 32 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.289937:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 186 00000100:00100000:3.0:1788845013.289957:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01998a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641336192:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845013.289977:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845013.290013:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 32 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.290036:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 32 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.290824:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ab727 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7276 expref: 32 pid: 11036 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.290846:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.290855:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.290869:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.290880:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.290954:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.290969:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660147712:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845013.291056:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.318643:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.318776:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641336320 00000100:00100000:0.0:1788845013.318788:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641336320 00002000:00100000:0.0:1788845013.318849:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360eb100 x1875739641336320/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 488/0 e 0 to 0 dl 1788845024 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845013.318861:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360eb100 with x1875739641336320 00000100:00100000:0.0:1788845013.319339:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 236 00000100:00100000:0.0:1788845013.319350:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641336320:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845013.319581:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641336320:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (947us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845013.319595:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 236 00002000:00100000:0.0:1788845013.319605:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360eb100 x1875739641336320/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 488/4536 e 0 to 0 dl 1788845024 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845013.319615:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360eb100 with x1875739641336320 00000100:00100000:0.0:1788845013.319790:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660147712:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845013.319813:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845013.320425:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01998a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641336192:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 30473us (32992us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845013.320435:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 186 00000100:00100000:1.0:1788845013.326011:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.326087:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641336448 00000100:00100000:3.0:1788845013.326096:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641336448 00000100:00100000:3.0:1788845013.326159:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 514 00000100:00100000:3.0:1788845013.326168:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2368:x1875739641336448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.326201:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.326221:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab75f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72a7 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.326258:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.326302:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660147840:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.326395:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463300 going to sleep for 11 seconds 00000004:00010000:3.0:1788845013.335311:0:5840:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845012 -> 1788845013 00000100:00100000:3.0:1788845013.335343:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660147840:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845013.335369:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:3.0:1788845013.335389:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab75f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a7 expref: 32 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.335409:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.335420:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845013.335426:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1c00/0xb2187301287ab75f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a7 expref: 32 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.335439:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1c00, rc 0) 00000100:00100000:3.0:1788845013.335511:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2368:x1875739641336448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9343us (9508us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845013.335520:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 514 00000100:00100000:0.0:1788845013.339909:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.339960:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641336576 00000100:00100000:3.0:1788845013.339971:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641336576 00000100:00100000:3.0:1788845013.340039:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 515 00000100:00100000:3.0:1788845013.340050:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0199b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+31:2368:x1875739641336576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.340071:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.340092:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab766 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72ae expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.340128:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.340169:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660147968:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.340273:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.346566:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660147968:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845013.346598:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:0.0:1788845013.346620:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab766 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72ae expref: 32 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.346642:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.346645:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845013.346651:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab766 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72ae expref: 32 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.346668:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1600, rc 0) 00000100:00100000:0.0:1788845013.346727:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0199b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+31:2368:x1875739641336576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6679us (6851us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845013.346736:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 515 00000100:00100000:3.0:1788845013.358509:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.365367:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641336704 00000100:00100000:0.0:1788845013.365392:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641336704 00002000:00100000:0.0:1788845013.365450:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035742a00 x1875739641336704/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 504/0 e 0 to 0 dl 1788845024 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845013.365462:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742a00 with x1875739641336704 00000100:00100000:0.0:1788845013.365521:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 237 00000100:00100000:0.0:1788845013.365532:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641336704:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845013.421268:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641336832 00000100:00100000:0.0:1788845013.421279:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641336832 00000100:00100000:0.0:1788845013.421341:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 516 00000100:00100000:0.0:1788845013.421353:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+31:2367:x1875739641336832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845013.421385:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845013.421406:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab76d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72b5 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.421446:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.421487:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0199ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660148096:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845013.421569:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000004:00010000:0.0:1788845013.432983:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845012 -> 1788845013 00000100:00100000:0.0:1788845013.433019:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0199ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660148096:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845013.433045:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:0.0:1788845013.433077:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab76d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72b5 expref: 32 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.433102:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.433106:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845013.433112:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab76d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72b5 expref: 32 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.433126:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17c00, rc 0) 00000100:00100000:0.0:1788845013.433204:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+31:2367:x1875739641336832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11853us (12020us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845013.433213:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 516 00000100:00100000:0.0:1788845013.450849:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.450906:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641336960 00000100:00100000:0.0:1788845013.450916:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641336960 00000100:00100000:0.0:1788845013.450980:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 517 00000100:00100000:0.0:1788845013.450990:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0199ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+31:2369:x1875739641336960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845013.451012:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845013.451032:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17400/0xb2187301287ab774 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72bc expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.451791:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.451841:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f825f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660148224:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845013.451951:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.457946:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f825f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660148224:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845013.457977:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:0.0:1788845013.457996:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17400/0xb2187301287ab774 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72bc expref: 32 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.458018:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.458022:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845013.458027:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17400/0xb2187301287ab774 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72bc expref: 32 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.458040:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17400, rc 0) 00000100:00100000:0.0:1788845013.459233:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0199ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+31:2369:x1875739641336960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8238us (8388us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845013.459244:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 517 00000100:00100000:0.0:1788845013.466855:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.466909:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641337088 00000100:00100000:0.0:1788845013.466924:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641337088 00000100:00100000:0.0:1788845013.466977:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 300 00000100:00100000:0.0:1788845013.466986:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641337088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845013.467008:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845013.467074:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab77b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72c3 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.467144:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660148352:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845013.467229:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00002000:00010000:0.0:1788845013.473272:0:6684:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845012 -> 1788845013 00002000:00010000:0.0:1788845013.473284:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845012 -> 1788845013 00000100:00100000:0.0:1788845013.473313:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660148352:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845013.473352:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab77b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72c3 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.473377:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000096ca43cc/0xb2187301287ab77b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72c3 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.473391:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d16c00, rc 0) 00000100:00100000:0.0:1788845013.473445:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641337088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6460us (6602us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845013.473454:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 300 00000100:00100000:3.0:1788845013.479386:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.483340:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641337216 00000100:00100000:1.0:1788845013.483354:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641337216 00000100:00100000:1.0:1788845013.483414:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 301 00000100:00100000:1.0:1788845013.483423:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641337216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.483441:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.483460:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ab782 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72ca expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.483516:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660148480:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845013.483604:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.503080:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.524720:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.528714:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641337472 00000100:00100000:0.0:1788845013.528731:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641337472 00000100:00100000:0.0:1788845013.528812:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 519 00000100:00100000:0.0:1788845013.528823:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641337472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845013.528858:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845013.528879:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab790 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72d8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.528964:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.529017:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660148736:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845013.529109:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.535498:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660148736:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845013.535533:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890784 00010000:00010000:0.0:1788845013.535558:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab790 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72d8 expref: 32 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.535581:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.535584:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845013.535589:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab790 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72d8 expref: 32 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.535605:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d16c00, rc 0) 00000100:00100000:0.0:1788845013.535684:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641337472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6862us (10983us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845013.535693:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 519 00000100:00100000:3.0:1788845013.541790:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.542000:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641337728 00000100:00100000:0.0:1788845013.542010:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641337728 00000100:00100000:0.0:1788845013.542068:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 520 00000100:00100000:0.0:1788845013.542077:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641337728:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845013.542145:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845013.542171:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1a00/0xb2187301287ab735 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a727d expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.542200:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.542205:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845013.542410:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.542424:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af200/0xb2187301287ab6d3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7222 expref: 30 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.542431:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.542462:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af200/0xb2187301287ab6d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7222 expref: 30 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.542482:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af200/0xb2187301287ab6d3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7222 expref: 30 pid: 5841 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.542495:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660148864:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845013.542571:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845013.554108:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.554138:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660148864:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845013.554168:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.554257:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641337856 00000100:00100000:1.0:1788845013.554264:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641337856 00010000:00010000:1.0:1788845013.554308:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af200/0xb2187301287ab6d3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7222 expref: 31 pid: 5841 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.554357:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 187 00000100:00100000:1.0:1788845013.554366:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082ff050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641337856:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845013.554379:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845013.554385:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af200/0xb2187301287ab6d3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7222 expref: 31 pid: 5841 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.554400:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af200/0xb2187301287ab6d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7222 expref: 31 pid: 5841 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.554420:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af200/0xb2187301287ab6d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7222 expref: 31 pid: 5841 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.554434:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845013.554439:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.554447:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.554454:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.554483:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845013.554530:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082ff050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+30:2370:x1875739641337856:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 165us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845013.554536:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 187 00010000:00010000:0.0:1788845013.560740:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.560768:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.560776:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.560782:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d16c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845013.560787:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.560794:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.560812:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.560815:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845013.560819:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.560825:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab797 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.564315:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.565245:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641337984 00000100:00100000:0.0:1788845013.565260:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641337984 00000100:00100000:0.0:1788845013.565518:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 302 00000100:00100000:0.0:1788845013.565528:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641337984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845013.565545:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845013.565562:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72df expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.565592:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1609727) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.565612:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:0.0:1788845013.565616:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:0.0:1788845013.565621:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.565782:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1609727) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.565800:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1609727) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 12 pid: 6682 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.565814:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660148992:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845013.575768:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.575864:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641338112 00000100:00100000:0.0:1788845013.575871:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641338112 00010000:00010000:0.0:1788845013.575938:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1609727) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 13 pid: 6682 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.575970:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 188 00000100:00100000:0.0:1788845013.575979:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5de680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641338112:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845013.575994:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845013.576016:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1609727) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 13 pid: 6682 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.576032:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1609727) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 13 pid: 6682 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.576071:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009be95aa9/0xb2187301287ab712 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1609727) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7261 expref: 13 pid: 6682 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.576086:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:0.0:1788845013.576089:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:0.0:1788845013.576096:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.576105:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.576137:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.576149:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660149120:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845013.576231:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00010000:00010000:1.0:1788845013.586102:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845013.587032:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660149120:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845013.587055:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845013.587110:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5de680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641338112:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11131us (11346us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845013.587120:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 188 00000100:00100000:1.0:1788845013.589088:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641338368 00000100:00100000:1.0:1788845013.589098:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641338368 00002000:00010000:1.0:1788845013.589147:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845013.589152:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c5500 with x1875739641338368 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845013.589164:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c5500. 00000100:00100000:1.0:1788845013.589197:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 143 00000100:00100000:1.0:1788845013.589206:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641338368:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845013.589478:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1659206 -> 0 00002000:00010000:1.0:1788845013.589482:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845013 -> 1788845013 00002000:00010000:1.0:1788845013.589485:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845013 -> 1788845010 00002000:00010000:1.0:1788845013.589487:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845013 -> 1788845013 00002000:00010000:1.0:1788845013.589489:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 259 -> 259 00000100:00100000:1.0:1788845013.589543:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641338368:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 338us (547us total) trans 4294967352 rc 0/0 00000100:00100000:1.0:1788845013.589551:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 143 00002000:00010000:1.0:1788845013.589554:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845013.589556:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c5500 with x1875739641338368 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845013.589560:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c5500. 00000100:00100000:1.0:1788845013.602744:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.602911:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641338240 00000100:00100000:0.0:1788845013.602921:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641338240 00002000:00010000:0.0:1788845013.602966:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845013.602984:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dce00 with x1875739641338240 00000100:00100000:0.0:1788845013.603013:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 238 00000100:00100000:0.0:1788845013.603021:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641338240:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845013.603875:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890784 -> 105895 00000004:00010000:0.0:1788845013.603880:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845013 -> 1788845013 00000004:00010000:0.0:1788845013.603905:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845012 -> 1788845013 00000004:00010000:0.0:1788845013.603907:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845013 -> 1788845013 00000004:00010000:0.0:1788845013.603909:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:0.0:1788845013.603970:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641338240:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 949us (1236us total) trans 4294967532 rc 0/0 00000100:00100000:0.0:1788845013.603978:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 238 00002000:00010000:0.0:1788845013.603982:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845013.603984:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dce00 with x1875739641338240 00000100:00100000:0.0:1788845013.630918:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641338496 00000100:00100000:0.0:1788845013.630928:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641338496 00000100:00100000:0.0:1788845013.630989:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 521 00000100:00100000:0.0:1788845013.630998:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+30:14891:x1875739641338496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845013.631019:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845013.631039:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab7a5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72e6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.634692:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.634718:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.634749:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845013.634792:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 105895 00000004:00200000:0.0:1788845013.634797:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845013.634816:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.634853:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16c00/0xb2187301287ab7a5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 32 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.634872:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 31 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.634880:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.634926:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845013.634931:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17800, rc 0) 00000100:00100000:0.0:1788845013.635000:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+31:14891:x1875739641338496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4001us (4188us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845013.635010:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 521 00000100:00100000:1.0:1788845013.640396:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641338752 00000100:00100000:1.0:1788845013.640406:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641338752 00000100:00100000:1.0:1788845013.640463:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 303 00000100:00100000:1.0:1788845013.640472:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641338752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.640488:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.640504:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ab7b3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72f4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.640571:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660149248:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845013.640655:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845013.640729:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641338624 00000100:00100000:3.0:1788845013.640734:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641338624 00000100:00100000:3.0:1788845013.640776:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 522 00000100:00100000:3.0:1788845013.640784:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2368:x1875739641338624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.640794:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.640807:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1e00/0xb2187301287ab7ba lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72ed expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.640837:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.640859:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660149376:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.640916:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845013.651692:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660149376:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845013.651716:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 105895 00010000:00010000:3.0:1788845013.651731:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1e00/0xb2187301287ab7ba lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72ed expref: 32 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.651746:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.651748:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845013.651751:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1e00/0xb2187301287ab7ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72ed expref: 32 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.651760:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1e00, rc 0) 00000100:00100000:3.0:1788845013.651800:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2368:x1875739641338624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11017us (11516us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845013.651808:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 522 00000100:00100000:0.0:1788845013.659404:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.659449:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.659534:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641338880 00000100:00100000:3.0:1788845013.659544:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641338880 00000100:00100000:3.0:1788845013.659603:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 523 00000100:00100000:3.0:1788845013.659613:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+31:2368:x1875739641338880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.659633:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.659653:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab7c8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a72fb expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.659691:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.659733:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891b800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660149504:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.659816:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845013.676171:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.676296:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641339264 00000100:00100000:1.0:1788845013.676306:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641339264 00000100:00100000:1.0:1788845013.676368:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 524 00000100:00100000:1.0:1788845013.676379:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641339264:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845013.676646:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.676664:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 30 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.676672:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.676704:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 30 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.676722:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 30 pid: 5839 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.676736:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660149760:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845013.676823:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845013.682468:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660149760:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845013.682501:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.687873:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.688212:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641339392 00000100:00100000:1.0:1788845013.688224:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641339392 00010000:00010000:1.0:1788845013.688350:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 31 pid: 5839 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.689947:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 189 00000100:00100000:1.0:1788845013.689970:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2369:x1875739641339392:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845013.689988:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845013.689999:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 31 pid: 5839 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.690021:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 31 pid: 5839 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.690046:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab7ac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72e6 expref: 31 pid: 5839 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.690065:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845013.690070:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.690081:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.690089:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.690117:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845013.690217:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+30:2369:x1875739641339392:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 256us (2353us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845013.690225:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 189 00010000:00010000:0.0:1788845013.693466:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.693482:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.693489:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.693507:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d15a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845013.693512:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.693519:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.693537:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.693540:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845013.693544:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.693550:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ab7cf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.694358:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.694398:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.695675:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641339520 00000100:00100000:0.0:1788845013.695692:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641339520 00002000:00010000:0.0:1788845013.695720:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845013.695724:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328380 with x1875739641339520 00000100:00100000:0.0:1788845013.695745:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 239 00000100:00100000:0.0:1788845013.695753:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641339520:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845013.695909:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 105895 -> 602056 00000004:00010000:0.0:1788845013.695913:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845013 -> 1788845013 00000004:00010000:0.0:1788845013.695915:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845013 -> 1788845013 00000004:00010000:0.0:1788845013.695918:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845013 -> 1788845013 00000004:00010000:0.0:1788845013.695920:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:0.0:1788845013.695968:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641339520:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 216us (1569us total) trans 4294967535 rc 0/0 00000100:00100000:0.0:1788845013.695988:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 239 00002000:00010000:0.0:1788845013.695991:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845013.695993:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328380 with x1875739641339520 00000100:00100000:3.0:1788845013.701089:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.701242:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641339776 00000100:00100000:0.0:1788845013.701253:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641339776 00002000:00100000:0.0:1788845013.701311:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d0e00 x1875739641339776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 488/0 e 0 to 0 dl 1788845024 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845013.701331:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d0e00 with x1875739641339776 00000100:00100000:0.0:1788845013.701363:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 240 00000100:00100000:0.0:1788845013.701373:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2369:x1875739641339776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845013.723243:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2369:x1875739641339776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 21860us (22154us total) trans 0 rc 495616/495616 00000100:00100000:1.0:1788845013.723256:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 240 00002000:00100000:1.0:1788845013.723269:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d0e00 x1875739641339776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 488/440 e 0 to 0 dl 1788845024 ref 1 fl Complete:/600/0 rc 495616/495616 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845013.723281:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d0e00 with x1875739641339776 00000100:00100000:0.0:1788845013.736924:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.737238:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641339904 00000100:00100000:1.0:1788845013.737249:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641339904 00002000:00100000:1.0:1788845013.737308:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c4b800 x1875739641339904/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 12720/0 e 0 to 0 dl 1788845024 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845013.737319:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c4b800 with x1875739641339904 00000100:00100000:1.0:1788845013.737361:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 241 00000100:00100000:1.0:1788845013.737372:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2368:x1875739641339904:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845013.737780:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 602056 -> 602056 00000004:00010000:1.0:1788845013.737784:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845013 -> 1788845013 00000004:00010000:1.0:1788845013.737787:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845013 -> 1788845013 00000004:00010000:1.0:1788845013.737789:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845013 -> 1788845013 00000004:00010000:1.0:1788845013.737791:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:1.0:1788845013.737864:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641339904:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 494us (990us total) trans 4294967536 rc 0/0 00000100:00100000:1.0:1788845013.737873:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 241 00002000:00100000:1.0:1788845013.737915:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c4b800 x1875739641339904/t4294967536(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 12720/448 e 0 to 0 dl 1788845024 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845013.737930:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c4b800 with x1875739641339904 00000100:00100000:0.0:1788845013.744624:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.812206:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641340032:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 67403us (67582us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845013.812223:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 525 00000100:00100000:0.0:1788845013.815128:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.815303:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641340160 00000100:00100000:3.0:1788845013.815313:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641340160 00000100:00100000:3.0:1788845013.815373:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 526 00000100:00100000:3.0:1788845013.815382:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641340160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.815401:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.815423:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab7d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7309 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.815509:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab7dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.815537:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab7dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.815551:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab7dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845013.815579:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 602056 00000004:00200000:3.0:1788845013.815583:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845013.815603:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab7dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.815638:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab7d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7309 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.815653:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab7dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7309 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.815661:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.815663:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845013.815666:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845013.815733:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641340160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 352us (612us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845013.815740:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 526 00000100:00100000:3.0:1788845013.826542:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.826630:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.827714:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641340416 00000100:00100000:1.0:1788845013.827720:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641340416 00000100:00100000:1.0:1788845013.827766:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 305 00000100:00100000:1.0:1788845013.827774:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641340416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.827785:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.827798:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ab7eb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7317 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.827843:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660150016:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845013.827904:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0031470c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845013.830716:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845010 -> 1788845013 00000100:00100000:1.0:1788845013.830748:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660150016:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845013.830787:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ab7eb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7317 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.830813:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ab7eb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7317 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.830826:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845013.830907:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641340416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3108us (4261us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.830929:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 305 00000100:00100000:0.0:1788845013.834502:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.834576:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641340672 00000100:00100000:1.0:1788845013.834585:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641340672 00000100:00100000:1.0:1788845013.834640:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 306 00000100:00100000:1.0:1788845013.834651:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+12:2367:x1875739641340672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.834670:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.834687:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ab7f2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7325 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.834752:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660150144:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845013.834832:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0031470c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.838101:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.838239:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641340544 00000100:00100000:3.0:1788845013.838247:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641340544 00000100:00100000:3.0:1788845013.838310:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 528 00000100:00100000:3.0:1788845013.838321:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641340544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.838357:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.838378:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab7f9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a731e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.838414:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.838453:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660150272:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.838532:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845013.843538:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660150272:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845013.843567:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602056 00010000:00010000:3.0:1788845013.843590:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab7f9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a731e expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.843612:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.843615:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845013.843620:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab7f9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a731e expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.843635:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1000, rc 0) 00000100:00100000:3.0:1788845013.843708:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641340544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5388us (5613us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845013.843718:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 528 00000100:00100000:1.0:1788845013.847287:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660150144:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845013.847336:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ab7f2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7325 expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.847362:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ab7f2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7325 expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.847377:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845013.847442:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+12:2367:x1875739641340672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12793us (12951us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845013.847450:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 306 00000100:00100000:0.0:1788845013.851072:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.851263:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641340800 00000100:00100000:3.0:1788845013.851280:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641340800 00000100:00100000:3.0:1788845013.851336:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 529 00000100:00100000:3.0:1788845013.851346:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641340800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.851381:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.851401:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab800 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a732c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.851431:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.851441:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab800 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a732c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.851476:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.851494:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 9 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.851508:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660150400:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845013.851592:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.861115:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.861958:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641340928 00000100:00100000:1.0:1788845013.861970:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641340928 00010000:00010000:1.0:1788845013.862040:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 10 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.862095:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 190 00000100:00100000:1.0:1788845013.862105:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641340928:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845013.862122:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845013.862149:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 10 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.862169:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 10 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.862201:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845013.862208:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab800 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a732c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.862220:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab800 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a732c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.862230:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab800 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a732c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.862285:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab800 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a732c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.862299:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660150528:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845013.862391:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0031470c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788845013.864030:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aec00/0xb2187301287ab758 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72a0 expref: 10 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.864068:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660150400:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845013.864097:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1000/0xb2187301287ab800 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a732c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.864114:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.864128:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845013.864132:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1000, rc 0) 00000100:00100000:3.0:1788845013.864204:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641340800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12859us (13149us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845013.864212:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 529 00000100:00100000:1.0:1788845013.872386:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660150528:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845013.872418:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845013.872536:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641340928:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10431us (11428us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845013.872547:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 190 00000100:00100000:0.0:1788845013.877828:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.877914:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641341056 00000100:00100000:1.0:1788845013.877921:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641341056 00002000:00100000:1.0:1788845013.877964:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c49c00 x1875739641341056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:119/0 lens 488/0 e 0 to 0 dl 1788845024 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845013.877973:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c49c00 with x1875739641341056 00000100:00100000:1.0:1788845013.878003:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 242 00000100:00100000:1.0:1788845013.878009:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641341056:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845013.892428:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641341056:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14407us (14597us total) trans 0 rc 602112/602112 00000100:00100000:1.0:1788845013.892441:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 242 00002000:00100000:1.0:1788845013.892459:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c49c00 x1875739641341056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:119/0 lens 488/440 e 0 to 0 dl 1788845024 ref 1 fl Complete:/600/0 rc 602112/602112 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845013.892469:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c49c00 with x1875739641341056 00000100:00100000:0.0:1788845013.895572:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.895677:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641341184 00000100:00100000:1.0:1788845013.895686:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641341184 00000100:00100000:1.0:1788845013.895756:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 530 00000100:00100000:1.0:1788845013.895766:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641341184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.895785:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.895805:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab807 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7333 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.895937:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.895951:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.895962:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845013.895990:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 602056 00000004:00200000:1.0:1788845013.895994:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845013.896010:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.896034:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab807 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.896060:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.896068:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845013.896071:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845013.896074:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845013.896120:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641341184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 357us (559us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845013.896126:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 530 00000100:00100000:3.0:1788845013.898567:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.898676:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641341312 00000100:00100000:0.0:1788845013.898685:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641341312 00000100:00100000:0.0:1788845013.898751:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 531 00000100:00100000:0.0:1788845013.898762:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641341312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845013.898783:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845013.898804:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab815 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a733a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.898855:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab815 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a733a expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845013.898867:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602056 00010000:00010000:0.0:1788845013.898874:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab815 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a733a expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.898909:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.898912:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845013.898915:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17c00, rc 0) 00000100:00100000:0.0:1788845013.898967:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641341312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 207us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845013.898973:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 531 00000100:00100000:0.0:1788845013.902626:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.908433:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.915563:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.915863:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641341696 00000100:00100000:1.0:1788845013.915910:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641341696 00002000:00100000:1.0:1788845013.915956:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b6680 x1875739641341696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 488/0 e 0 to 0 dl 1788845024 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845013.915966:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6680 with x1875739641341696 00000100:00100000:1.0:1788845013.915996:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 243 00000100:00100000:1.0:1788845013.916004:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641341696:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845013.930389:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641341696:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14375us (14826us total) trans 0 rc 602112/602112 00000100:00100000:1.0:1788845013.930402:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 243 00002000:00100000:1.0:1788845013.930416:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b6680 x1875739641341696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:119/0 lens 488/440 e 0 to 0 dl 1788845024 ref 1 fl Complete:/600/0 rc 602112/602112 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845013.930426:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6680 with x1875739641341696 00000100:00100000:1.0:1788845013.935212:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.938447:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660150912:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845013.938486:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.938506:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845013.938510:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845013.938515:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.938526:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023af000, rc 0) 00000100:00100000:1.0:1788845013.938595:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641341824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3197us (3392us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845013.938622:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 532 00000100:00100000:1.0:1788845013.943536:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.943635:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641341952 00000100:00100000:0.0:1788845013.943646:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641341952 00010000:00010000:0.0:1788845013.943690:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab815 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a733a expref: 11 pid: 11036 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.943740:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 191 00000100:00100000:0.0:1788845013.943751:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641341952:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845013.943764:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845013.943790:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab815 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a733a expref: 11 pid: 11036 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.943807:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab815 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a733a expref: 11 pid: 11036 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.943843:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab815 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a733a expref: 11 pid: 11036 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.943860:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.943866:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.943876:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.943936:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.943976:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.943991:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660151040:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845013.944088:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.946721:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660151040:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845013.946747:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845013.946809:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641341952:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3058us (3280us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845013.946819:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 191 00000100:00100000:3.0:1788845013.949553:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.949637:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641342080 00000100:00100000:0.0:1788845013.949647:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641342080 00000100:00100000:0.0:1788845013.949742:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 244 00000100:00100000:0.0:1788845013.949751:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641342080:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845013.950162:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 602056 -> 602056 00000004:00010000:0.0:1788845013.950166:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845013 -> 1788845013 00000004:00010000:0.0:1788845013.950168:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845013 -> 1788845013 00000004:00010000:0.0:1788845013.950170:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845013 -> 1788845013 00000004:00010000:0.0:1788845013.950181:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 262 00000100:00100000:0.0:1788845013.950254:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641342080:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 503us (709us total) trans 4294967537 rc 0/0 00000100:00100000:0.0:1788845013.950261:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 244 00000100:00100000:1.0:1788845013.952959:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.953048:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641342208 00000100:00100000:0.0:1788845013.953068:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641342208 00000100:00100000:0.0:1788845013.953130:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 533 00000100:00100000:0.0:1788845013.953140:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641342208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845013.953162:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845013.953194:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab831 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7356 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.953232:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.953271:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660151168:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845013.953359:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.955731:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660151168:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845013.955759:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602056 00010000:00010000:0.0:1788845013.955795:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab831 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7356 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.955816:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.955819:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845013.955824:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab831 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7356 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.955839:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17c00, rc 0) 00000100:00100000:0.0:1788845013.955941:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641342208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2802us (2989us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845013.955949:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 533 00000100:00100000:3.0:1788845013.958815:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641342336 00000100:00100000:3.0:1788845013.958825:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641342336 00000100:00100000:3.0:1788845013.958912:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 534 00000100:00100000:3.0:1788845013.958922:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641342336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.958941:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.958960:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0c00/0xb2187301287ab838 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a735d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.958993:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.959029:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660151296:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.959118:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845013.962072:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660151296:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845013.962100:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602056 00010000:00010000:3.0:1788845013.962118:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0c00/0xb2187301287ab838 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a735d expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.962138:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.962153:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845013.962157:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0c00/0xb2187301287ab838 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a735d expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.962182:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0c00, rc 0) 00000100:00100000:3.0:1788845013.962241:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641342336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3320us (3522us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845013.962250:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 534 00000100:00100000:0.0:1788845013.965338:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.965488:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641342592 00000100:00100000:3.0:1788845013.965497:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641342592 00000100:00100000:3.0:1788845013.965556:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 535 00000100:00100000:3.0:1788845013.965565:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641342592:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845013.965631:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845013.965657:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab7dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7309 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.965676:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.965680:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845013.965877:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.965922:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.965931:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.965961:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.965997:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 9 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.966011:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660151424:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845013.966094:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.969791:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641342720 00000100:00100000:0.0:1788845013.969801:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641342720 00010000:00010000:0.0:1788845013.969840:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 10 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.969921:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 192 00000100:00100000:0.0:1788845013.969932:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641342720:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845013.969946:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845013.969953:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 10 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.969971:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 10 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.969994:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.970012:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.970022:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.970028:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.970058:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845013.970112:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641342720:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 181us (475us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845013.970120:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 192 00010000:00010000:3.0:1788845013.970837:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ab80e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7333 expref: 9 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.970857:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660151424:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845013.970898:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.970908:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.970915:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845013.971032:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967538 00000004:00010000:3.0:1788845013.971085:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 602056 -> 602056 00000004:00010000:3.0:1788845013.971089:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845013 -> 1788845013 00000004:00010000:3.0:1788845013.971091:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845013 -> 1788845013 00000004:00010000:3.0:1788845013.971093:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845013 -> 1788845013 00000004:00010000:3.0:1788845013.971094:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 262 -> 262 00000004:00010000:3.0:1788845013.971101:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004f82dbf2 reply state 00000000219e455d transno 4294967539 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.971122:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845013.971215:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641342592:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5650us (5886us total) trans 4294967539 rc 0/0 00000100:00100000:3.0:1788845013.971221:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 535 00000020:00080000:3.0:1788845013.971225:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845013.974444:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.974683:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641342848 00000100:00100000:1.0:1788845013.974700:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641342848 00000100:00100000:1.0:1788845013.974758:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 309 00000100:00100000:1.0:1788845013.974767:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:14891:x1875739641342848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845013.974786:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845013.974804:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7364 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.974835:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 8 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.974843:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(5 11) 00010000:00010000:1.0:1788845013.974847:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:1.0:1788845013.974853:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.974916:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 8 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845013.974934:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 8 pid: 7983 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.974969:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5ddc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660151552:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845013.975062:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147840 going to sleep for 11 seconds 00010000:00010000:0.0:1788845013.975623:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.975632:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.975639:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.975644:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01cb1600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845013.975649:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.975656:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.975672:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845013.975675:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845013.975679:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.975703:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1600/0xb2187301287ab83f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845013.980732:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.980808:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641342976 00000100:00100000:0.0:1788845013.980817:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641342976 00010000:00010000:0.0:1788845013.980856:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 9 pid: 7983 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.980939:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 193 00000100:00100000:0.0:1788845013.980949:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641342976:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845013.980965:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845013.980990:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 9 pid: 7983 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.981008:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 9 pid: 7983 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.981032:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b477aa45/0xb2187301287ab79e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a72df expref: 9 pid: 7983 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.981048:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:0.0:1788845013.981064:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:0.0:1788845013.981074:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.981105:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845013.981140:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845013.981152:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660151680:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845013.981230:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845013.983416:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660151680:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845013.983440:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845013.983483:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641342976:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2536us (2760us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845013.983493:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 193 00000100:00100000:1.0:1788845013.985964:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.985996:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845013.986100:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641343232 00000100:00100000:0.0:1788845013.986111:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641343232 00002000:00010000:0.0:1788845013.986153:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845013.986157:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5ddc00 with x1875739641343232 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845013.986170:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5ddc00. 00000100:00100000:0.0:1788845013.986214:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 145 00000100:00100000:0.0:1788845013.986223:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641343232:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845013.986471:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1788845013.986475:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845013 -> 1788845013 00002000:00010000:0.0:1788845013.986477:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845013 -> 1788845010 00002000:00010000:0.0:1788845013.986493:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845013 -> 1788845013 00002000:00010000:0.0:1788845013.986495:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 259 -> 259 00000100:00100000:0.0:1788845013.986545:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641343232:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 323us (547us total) trans 4294967354 rc 0/0 00000100:00100000:0.0:1788845013.986552:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 145 00002000:00010000:0.0:1788845013.986555:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845013.986557:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5ddc00 with x1875739641343232 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845013.986562:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5ddc00. 00000100:00100000:3.0:1788845013.989629:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.993207:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845013.993248:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845013.993362:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641343616 00000100:00100000:3.0:1788845013.993372:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641343616 00000100:00100000:3.0:1788845013.993425:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 310 00000100:00100000:3.0:1788845013.993433:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641343616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845013.993451:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845013.993467:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab85b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7379 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845013.993522:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660151808:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845013.993592:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00002000:00010000:3.0:1788845013.996343:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845010 -> 1788845013 00000100:00100000:3.0:1788845013.996372:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660151808:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845013.996408:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab85b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7379 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.996431:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a70896bd/0xb2187301287ab85b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7379 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.996463:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845013.996521:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641343616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3088us (3271us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845013.996529:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 310 00000100:00100000:3.0:1788845013.999768:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660151936:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845013.999794:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 812324 00010000:00010000:3.0:1788845013.999814:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae600/0xb2187301287ab862 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7372 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.999833:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845013.999835:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845013.999840:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023ae600/0xb2187301287ab862 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7372 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845013.999853:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023ae600, rc 0) 00000100:00100000:3.0:1788845014.000076:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641343488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6246us (6877us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845014.000084:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 537 00000100:00100000:1.0:1788845014.002866:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.002951:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.003733:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641343872 00000100:00100000:3.0:1788845014.003738:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641343872 00000100:00100000:3.0:1788845014.003779:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 311 00000100:00100000:3.0:1788845014.003787:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641343872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.003797:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.003810:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b3fc064/0xb2187301287ab870 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7387 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.003869:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660152192:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845014.004210:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845014.013482:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.013567:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.013669:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641344000 00000100:00100000:3.0:1788845014.013681:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641344000 00000100:00100000:3.0:1788845014.013759:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:3.0:1788845014.013770:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641344000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.013795:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.013818:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab877 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a738e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.013858:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.013932:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660152320:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845014.014011:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.019641:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660152320:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845014.019659:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 812324 00010000:00010000:3.0:1788845014.019673:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab877 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a738e expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.019687:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845014.019691:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845014.019696:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab877 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a738e expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.019740:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845014.019789:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641344000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6021us (6319us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845014.019798:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:1.0:1788845014.023921:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.024056:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641344256 00000100:00100000:0.0:1788845014.024066:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641344256 00000100:00100000:0.0:1788845014.024115:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 540 00000100:00100000:0.0:1788845014.024125:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01998a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641344256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845014.024145:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845014.024166:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a739c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.024210:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.024218:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a739c expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.024248:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.024263:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 17 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.024295:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660152576:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845014.024376:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.026857:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660152576:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845014.026926:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a739c expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.026944:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.026947:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845014.026951:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a739c expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.026960:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17a00, rc 0) 00000100:00100000:0.0:1788845014.027005:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01998a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641344256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2882us (3133us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845014.027014:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 540 00000100:00100000:3.0:1788845014.029782:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.029939:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641344384 00000100:00100000:0.0:1788845014.029951:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641344384 00010000:00010000:0.0:1788845014.030001:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 18 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.030061:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 194 00000100:00100000:0.0:1788845014.030073:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082ffc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641344384:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845014.030087:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845014.030120:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 18 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.030156:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 18 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.030190:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab82a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a734f expref: 18 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.030207:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.030213:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a739c expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.030222:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a739c expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.030231:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a739c expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.030272:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a739c expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.030287:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660152704:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845014.030375:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.033518:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660152704:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845014.033544:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845014.033600:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082ffc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641344384:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3529us (3832us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845014.033627:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 194 00000100:00100000:0.0:1788845014.035395:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641344512 00000100:00100000:0.0:1788845014.035404:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641344512 00002000:00100000:0.0:1788845014.035452:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d500 x1875739641344512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 504/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.035464:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d500 with x1875739641344512 00000100:00100000:0.0:1788845014.035502:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 246 00000100:00100000:0.0:1788845014.035511:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641344512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.041030:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641344512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 5508us (5695us total) trans 0 rc 217088/217088 00000100:00100000:0.0:1788845014.041051:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 246 00002000:00100000:0.0:1788845014.041067:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d500 x1875739641344512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 504/440 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 217088/217088 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.041076:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d500 with x1875739641344512 00000100:00100000:0.0:1788845014.044312:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641344640 00000100:00100000:0.0:1788845014.044322:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641344640 00002000:00100000:0.0:1788845014.044368:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ce00 x1875739641344640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.044379:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ce00 with x1875739641344640 00000100:00100000:0.0:1788845014.044463:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 247 00000100:00100000:0.0:1788845014.044473:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641344640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.044750:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641344640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 278us (550us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845014.044757:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 247 00002000:00100000:0.0:1788845014.044779:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ce00 x1875739641344640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.044787:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ce00 with x1875739641344640 00000100:00100000:3.0:1788845014.047507:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.048410:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641344768 00000100:00100000:1.0:1788845014.048421:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641344768 00002000:00100000:1.0:1788845014.048472:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b6a00 x1875739641344768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845014.048482:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6a00 with x1875739641344768 00000100:00100000:1.0:1788845014.048520:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 248 00000100:00100000:1.0:1788845014.048528:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641344768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845014.048803:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641344768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (1304us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845014.048811:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 248 00002000:00100000:1.0:1788845014.048819:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b6a00 x1875739641344768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845014.048847:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6a00 with x1875739641344768 00000100:00100000:1.0:1788845014.051953:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.052112:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641344896 00000100:00100000:0.0:1788845014.052122:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641344896 00002000:00100000:0.0:1788845014.052181:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657100 x1875739641344896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.052191:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657100 with x1875739641344896 00000100:00100000:0.0:1788845014.052263:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 249 00000100:00100000:0.0:1788845014.052273:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641344896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.052553:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641344896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (609us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845014.052560:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 249 00002000:00100000:0.0:1788845014.052567:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657100 x1875739641344896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.052574:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657100 with x1875739641344896 00000100:00100000:0.0:1788845014.055854:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641345024 00000100:00100000:0.0:1788845014.055865:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641345024 00002000:00100000:0.0:1788845014.055932:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481f80 x1875739641345024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.055944:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481f80 with x1875739641345024 00000100:00100000:0.0:1788845014.056009:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 250 00000100:00100000:0.0:1788845014.056020:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641345024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.056387:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641345024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 368us (699us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845014.056405:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 250 00002000:00100000:0.0:1788845014.056413:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481f80 x1875739641345024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.056421:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481f80 with x1875739641345024 00000100:00100000:1.0:1788845014.059632:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.059778:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641345152 00000100:00100000:3.0:1788845014.059787:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641345152 00002000:00100000:3.0:1788845014.059841:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f655f80 x1875739641345152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.059853:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f655f80 with x1875739641345152 00000100:00100000:3.0:1788845014.059938:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 251 00000100:00100000:3.0:1788845014.059948:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641345152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.060210:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641345152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 262us (588us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845014.060217:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 251 00002000:00100000:3.0:1788845014.060224:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f655f80 x1875739641345152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.060231:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f655f80 with x1875739641345152 00000100:00100000:1.0:1788845014.063425:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.067420:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641345408 00000100:00100000:0.0:1788845014.067429:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641345408 00002000:00100000:0.0:1788845014.067479:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482a00 x1875739641345408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.067489:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482a00 with x1875739641345408 00000100:00100000:0.0:1788845014.067525:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 253 00000100:00100000:0.0:1788845014.067534:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641345408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.067797:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641345408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 262us (485us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845014.067804:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 253 00002000:00100000:0.0:1788845014.067811:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482a00 x1875739641345408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.067819:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482a00 with x1875739641345408 00000100:00100000:1.0:1788845014.070686:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.074766:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.074915:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641345664 00000100:00100000:0.0:1788845014.074924:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641345664 00002000:00100000:0.0:1788845014.074975:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656a00 x1875739641345664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.074986:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656a00 with x1875739641345664 00000100:00100000:0.0:1788845014.075025:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 255 00000100:00100000:0.0:1788845014.075035:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641345664:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.075341:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641345664:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 306us (584us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845014.075349:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 255 00002000:00100000:0.0:1788845014.075356:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656a00 x1875739641345664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.075363:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656a00 with x1875739641345664 00000100:00100000:3.0:1788845014.078419:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.082642:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.082736:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641345920 00000100:00100000:0.0:1788845014.082744:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641345920 00002000:00100000:0.0:1788845014.082796:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657480 x1875739641345920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.082807:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657480 with x1875739641345920 00000100:00100000:0.0:1788845014.082845:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 257 00000100:00100000:0.0:1788845014.082854:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641345920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.083139:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641345920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 285us (505us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845014.083146:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 257 00002000:00100000:0.0:1788845014.083152:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657480 x1875739641345920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.083159:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657480 with x1875739641345920 00000100:00100000:1.0:1788845014.086405:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.090599:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641346176 00000100:00100000:0.0:1788845014.090611:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641346176 00002000:00100000:0.0:1788845014.090660:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d5f80 x1875739641346176/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.090672:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d5f80 with x1875739641346176 00000100:00100000:0.0:1788845014.090732:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 259 00000100:00100000:0.0:1788845014.090743:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641346176:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.091024:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641346176:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 282us (574us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845014.091032:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 259 00002000:00100000:0.0:1788845014.091038:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d5f80 x1875739641346176/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.091057:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d5f80 with x1875739641346176 00000100:00100000:1.0:1788845014.094092:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.097488:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.100439:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.104421:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.104535:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641346688 00000100:00100000:0.0:1788845014.104544:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641346688 00000100:00100000:0.0:1788845014.104602:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 541 00000100:00100000:0.0:1788845014.104612:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641346688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845014.104637:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845014.104658:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab88c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73a3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.104757:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17600/0xb2187301287ab893 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.104773:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17600/0xb2187301287ab893 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.104784:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17600/0xb2187301287ab893 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845014.104810:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 812324 00000004:00200000:0.0:1788845014.104814:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845014.104831:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17600/0xb2187301287ab893 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.104869:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17c00/0xb2187301287ab88c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73a3 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.105123:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17600/0xb2187301287ab893 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73a3 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.105132:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.105134:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845014.105138:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17600, rc 0) 00000100:00100000:1.0:1788845014.105259:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641346688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 647us (847us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845014.105266:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 541 00000100:00100000:1.0:1788845014.107866:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.114163:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641346944 00000100:00100000:0.0:1788845014.114184:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641346944 00002000:00100000:0.0:1788845014.114242:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036080000 x1875739641346944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.114255:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036080000 with x1875739641346944 00000100:00100000:0.0:1788845014.114300:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 263 00000100:00100000:0.0:1788845014.114310:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641346944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.139583:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641346944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 25263us (25565us total) trans 0 rc 532480/532480 00000100:00100000:0.0:1788845014.139596:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 263 00002000:00100000:0.0:1788845014.139610:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036080000 x1875739641346944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/440 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 532480/532480 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.139620:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036080000 with x1875739641346944 00000100:00100000:1.0:1788845014.164878:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.179979:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.180521:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641347200 00000100:00100000:1.0:1788845014.180532:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641347200 00000100:00100000:1.0:1788845014.180595:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 314 00000100:00100000:1.0:1788845014.180605:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641347200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845014.180624:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845014.180641:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ef1f9f8/0xb2187301287ab8a8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73b8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845014.180712:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660152960:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845014.180816:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.182966:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660152960:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845014.183021:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ef1f9f8/0xb2187301287ab8a8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73b8 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.183058:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ef1f9f8/0xb2187301287ab8a8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73b8 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.183072:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3400, rc 0) 00000100:00100000:3.0:1788845014.183127:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641347200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2523us (3157us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845014.183135:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 314 00000100:00100000:3.0:1788845014.187986:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.188088:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641347328 00000100:00100000:0.0:1788845014.188098:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641347328 00000100:00100000:0.0:1788845014.188150:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 315 00000100:00100000:0.0:1788845014.188184:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641347328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845014.188203:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845014.188220:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000595e6663/0xb2187301287ab8af lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73bf expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.188277:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660153088:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845014.188362:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.203727:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660153088:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845014.203769:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000595e6663/0xb2187301287ab8af lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73bf expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.203793:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000595e6663/0xb2187301287ab8af lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73bf expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.203808:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17c00, rc 0) 00000100:00100000:0.0:1788845014.203873:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641347328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15702us (15897us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845014.203934:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 315 00000100:00100000:1.0:1788845014.207407:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.215219:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.215352:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641347584 00000100:00100000:1.0:1788845014.215361:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641347584 00000100:00100000:1.0:1788845014.215414:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 317 00000100:00100000:1.0:1788845014.215423:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641347584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845014.215442:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845014.215460:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000be4f14fb/0xb2187301287ab8bd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73cd expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845014.215516:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660153344:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845014.215598:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.230929:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.231091:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641347712 00000100:00100000:1.0:1788845014.231102:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641347712 00000100:00100000:1.0:1788845014.231180:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 318 00000100:00100000:1.0:1788845014.231190:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641347712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845014.231208:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845014.231226:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c182c1e6/0xb2187301287ab8c4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73d4 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845014.231283:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660153472:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845014.231363:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845014.234608:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660153472:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845014.234650:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c182c1e6/0xb2187301287ab8c4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73d4 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.234704:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c182c1e6/0xb2187301287ab8c4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73d4 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.234716:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3a00, rc 0) 00000100:00100000:1.0:1788845014.234777:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641347712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3589us (3903us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845014.234785:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 318 00000100:00100000:3.0:1788845014.239379:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.239467:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641347840 00000100:00100000:0.0:1788845014.239478:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641347840 00002000:00100000:0.0:1788845014.239525:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5f80 x1875739641347840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.239535:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5f80 with x1875739641347840 00000100:00100000:0.0:1788845014.239564:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 264 00000100:00100000:0.0:1788845014.239588:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641347840:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.239844:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641347840:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (473us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845014.239852:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 264 00002000:00100000:0.0:1788845014.239858:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5f80 x1875739641347840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/440 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.239865:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5f80 with x1875739641347840 00000100:00100000:3.0:1788845014.243479:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.243984:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641347968 00000100:00100000:0.0:1788845014.243995:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641347968 00000100:00100000:0.0:1788845014.244187:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 319 00000100:00100000:0.0:1788845014.244198:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641347968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845014.244217:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845014.244236:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e836f582/0xb2187301287ab8cb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73db expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.244296:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660153600:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845014.244382:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.247250:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660153600:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845014.247377:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e836f582/0xb2187301287ab8cb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73db expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.247403:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e836f582/0xb2187301287ab8cb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73db expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.247438:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17400, rc 0) 00000100:00100000:0.0:1788845014.247511:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641347968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3313us (4039us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845014.247519:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 319 00000100:00100000:3.0:1788845014.250820:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.267058:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.272123:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660153856:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845014.272180:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d1e478de/0xb2187301287ab8d9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73e9 expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.272207:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d1e478de/0xb2187301287ab8d9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73e9 expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.272222:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023aec00, rc 0) 00000100:00100000:1.0:1788845014.272484:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641348224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5179us (5444us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845014.272493:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 321 00000100:00100000:1.0:1788845014.287591:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.291038:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641348352 00000100:00100000:3.0:1788845014.291118:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641348352 00000100:00100000:3.0:1788845014.291250:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 543 00000100:00100000:3.0:1788845014.291261:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641348352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.291285:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.291289:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845014.291342:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17a00/0xb2187301287ab885 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a739c expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.291360:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845014.291364:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845014.291392:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73f0 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.291417:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afe00/0xb2187301287ab89a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73aa expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.291427:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.291641:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afe00/0xb2187301287ab89a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73aa expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.291663:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afe00/0xb2187301287ab89a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73aa expref: 18 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.291820:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660153984:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845014.292019:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.299041:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.299160:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660153984:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845014.299209:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.299229:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845014.299233:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845014.299239:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.299249:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0600, rc 0) 00000100:00100000:3.0:1788845014.299333:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641348352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8074us (11753us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845014.299342:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 543 00000100:00100000:3.0:1788845014.300206:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641348480 00000100:00100000:3.0:1788845014.300216:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641348480 00010000:00010000:3.0:1788845014.300254:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afe00/0xb2187301287ab89a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73aa expref: 19 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.300288:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 195 00000100:00100000:3.0:1788845014.300297:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641348480:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845014.300311:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845014.300336:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afe00/0xb2187301287ab89a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73aa expref: 19 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.300353:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afe00/0xb2187301287ab89a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73aa expref: 19 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.300374:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afe00/0xb2187301287ab89a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73aa expref: 19 pid: 5841 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.300388:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845014.300394:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.300404:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.300429:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.300468:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.300483:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660154112:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845014.300542:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.309089:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660154112:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845014.309117:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845014.309180:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641348480:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8882us (10149us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845014.309192:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 195 00000100:00100000:0.0:1788845014.320261:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.320356:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641348608 00000100:00100000:3.0:1788845014.320367:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641348608 00002000:00100000:3.0:1788845014.320422:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c4a680 x1875739641348608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.320434:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c4a680 with x1875739641348608 00000100:00100000:3.0:1788845014.320469:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 265 00000100:00100000:3.0:1788845014.320479:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641348608:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.320739:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641348608:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 260us (491us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845014.320747:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 265 00002000:00100000:3.0:1788845014.320754:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c4a680 x1875739641348608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.320774:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c4a680 with x1875739641348608 00000100:00100000:0.0:1788845014.323924:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.331970:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.332054:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641348864 00000100:00100000:3.0:1788845014.332064:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641348864 00002000:00100000:3.0:1788845014.332112:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8380 x1875739641348864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.332122:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8380 with x1875739641348864 00000100:00100000:3.0:1788845014.332149:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 267 00000100:00100000:3.0:1788845014.332157:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641348864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.332433:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641348864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (472us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845014.332441:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 267 00002000:00100000:3.0:1788845014.332449:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8380 x1875739641348864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.332457:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8380 with x1875739641348864 00000100:00100000:0.0:1788845014.340119:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.348785:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.348879:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641349120 00000100:00100000:3.0:1788845014.348924:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641349120 00002000:00100000:3.0:1788845014.348974:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efb800 x1875739641349120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.348985:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efb800 with x1875739641349120 00000100:00100000:3.0:1788845014.349013:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 269 00000100:00100000:3.0:1788845014.349022:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641349120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.349341:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641349120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 318us (565us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845014.349360:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 269 00002000:00100000:3.0:1788845014.349368:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efb800 x1875739641349120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.349376:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efb800 with x1875739641349120 00000100:00100000:3.0:1788845014.358964:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.363298:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.363418:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641349376 00000100:00100000:3.0:1788845014.363427:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641349376 00002000:00100000:3.0:1788845014.363476:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efa300 x1875739641349376/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.363486:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efa300 with x1875739641349376 00000100:00100000:3.0:1788845014.363524:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 271 00000100:00100000:3.0:1788845014.363533:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641349376:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845014.387030:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 812324 -> 812324 00000004:00010000:3.0:1788845014.387047:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845013 -> 1788845014 00000004:00010000:3.0:1788845014.387050:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845013 -> 1788845013 00000004:00010000:3.0:1788845014.387053:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845013 -> 1788845014 00000004:00010000:3.0:1788845014.387055:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 262 -> 262 00000100:00100000:3.0:1788845014.387136:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641349376:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 23598us (23842us total) trans 4294967541 rc 0/0 00000100:00100000:3.0:1788845014.387147:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 271 00002000:00100000:3.0:1788845014.387161:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efa300 x1875739641349376/t4294967541(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/448 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.387181:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efa300 with x1875739641349376 00000100:00100000:0.0:1788845014.406640:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.406753:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641349504 00000100:00100000:3.0:1788845014.406782:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641349504 00000100:00100000:3.0:1788845014.406855:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 544 00000100:00100000:3.0:1788845014.406868:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641349504:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:0.0:1788845014.510631:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641349504:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 103753us (103993us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845014.510648:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 544 00000100:00100000:0.0:1788845014.526654:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.526793:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641349632 00000100:00100000:1.0:1788845014.526818:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641349632 00000100:00100000:1.0:1788845014.526878:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 545 00000100:00100000:1.0:1788845014.526908:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641349632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845014.526932:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845014.526955:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3000/0xb2187301287ab8e7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73f7 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.526996:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845014.527039:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660154240:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845014.527942:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147cc0 going to sleep for 11 seconds 00000004:00010000:0.0:1788845014.541880:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845013 -> 1788845014 00000004:00010000:0.0:1788845014.541907:0:11036:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 262 -> 390 00000100:00100000:0.0:1788845014.541938:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660154240:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845014.541970:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 812324 00010000:00010000:0.0:1788845014.541990:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3000/0xb2187301287ab8e7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.542011:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.542014:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845014.542020:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3000/0xb2187301287ab8e7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.542128:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3000, rc 0) 00000100:00100000:0.0:1788845014.542288:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641349632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15382us (15644us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845014.542297:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 545 00000100:00100000:3.0:1788845014.553871:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641349760 00000100:00100000:3.0:1788845014.553907:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641349760 00000100:00100000:3.0:1788845014.553971:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 546 00000100:00100000:3.0:1788845014.553983:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f7ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641349760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.554004:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.554025:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0a00/0xb2187301287ab8ee lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a73fe expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.554077:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.554120:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660154368:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845014.554220:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.569650:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641349888 00000100:00100000:3.0:1788845014.569662:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641349888 00000100:00100000:3.0:1788845014.569748:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 547 00000100:00100000:3.0:1788845014.569761:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641349888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.569782:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.569804:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab8f5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7405 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.569843:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.569954:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660154496:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845014.570035:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845014.583224:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660154496:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845014.583257:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 812324 00010000:00010000:1.0:1788845014.583278:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab8f5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7405 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.583300:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845014.583303:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845014.583308:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0200/0xb2187301287ab8f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7405 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.583324:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0200, rc 0) 00000100:00100000:1.0:1788845014.583397:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641349888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13639us (13848us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845014.583406:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 547 00000100:00100000:1.0:1788845014.593227:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.596651:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f777b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660154624:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845014.596694:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 812324 00010000:00010000:3.0:1788845014.596714:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aee00/0xb2187301287ab8fc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a740c expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.596734:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845014.596737:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845014.596741:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aee00/0xb2187301287ab8fc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a740c expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.596755:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023aee00, rc 0) 00000100:00100000:3.0:1788845014.596812:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641350016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3349us (3595us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845014.596820:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 548 00000100:00100000:3.0:1788845014.599987:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.600078:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641350144 00000100:00100000:0.0:1788845014.600088:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641350144 00000100:00100000:0.0:1788845014.600144:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 549 00000100:00100000:0.0:1788845014.600154:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641350144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845014.600184:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845014.600204:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16000/0xb2187301287ab903 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7413 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.600237:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.600276:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660154752:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845014.600355:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0182b3f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.603254:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660154752:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845014.603280:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 812324 00010000:00010000:0.0:1788845014.603298:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16000/0xb2187301287ab903 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7413 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.603318:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.603321:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845014.603326:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d16000/0xb2187301287ab903 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7413 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.603355:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d16000, rc 0) 00000100:00100000:0.0:1788845014.603414:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641350144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3260us (3435us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845014.603422:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 549 00000100:00100000:0.0:1788845014.606612:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641350272 00000100:00100000:0.0:1788845014.606621:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641350272 00000100:00100000:0.0:1788845014.606679:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 550 00000100:00100000:0.0:1788845014.606692:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641350272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845014.606712:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845014.606730:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab90a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a741a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.606762:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.606795:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660154880:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845014.606869:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0182b3f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.609309:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660154880:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845014.609335:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 812324 00010000:00010000:0.0:1788845014.609356:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab90a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a741a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.609377:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.609381:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845014.609385:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17000/0xb2187301287ab90a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a741a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.609418:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17000, rc 0) 00000100:00100000:0.0:1788845014.609481:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641350272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2788us (2948us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845014.609490:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 550 00010000:00010000:0.0:1788845014.610106:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845014.613426:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.613550:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641350400 00000100:00100000:3.0:1788845014.613561:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641350400 00000100:00100000:3.0:1788845014.613617:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 551 00000100:00100000:3.0:1788845014.613627:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641350400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.613650:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.613686:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7421 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.613728:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.613736:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.613768:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.613784:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 10 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.613796:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660155008:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845014.613877:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.616230:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660155008:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845014.616266:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.616285:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845014.616288:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845014.616292:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.616301:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb1200, rc 0) 00000100:00100000:3.0:1788845014.616389:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641350400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2764us (2973us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845014.616409:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 551 00000100:00100000:1.0:1788845014.619516:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.620584:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641350528 00000100:00100000:0.0:1788845014.620596:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641350528 00010000:00010000:0.0:1788845014.620648:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 11 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.620709:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 196 00000100:00100000:0.0:1788845014.620718:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641350528:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845014.620734:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845014.620759:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 11 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.620777:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 11 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.620798:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0600/0xb2187301287ab8e0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a73f0 expref: 11 pid: 5839 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.620815:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.620822:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.620851:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.620859:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.620939:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845014.620953:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660155136:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845014.621141:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0182b3f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.627644:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641350656 00000100:00100000:3.0:1788845014.627652:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641350656 00000100:00100000:3.0:1788845014.627757:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 272 00000100:00100000:3.0:1788845014.627767:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641350656:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845014.628030:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 812324 -> 812324 00000004:00010000:3.0:1788845014.628034:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845014 -> 1788845014 00000004:00010000:3.0:1788845014.628438:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845014 -> 1788845014 00000004:00010000:3.0:1788845014.628441:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845014 -> 1788845014 00000004:00010000:3.0:1788845014.628443:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:3.0:1788845014.628514:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641350656:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 747us (954us total) trans 4294967542 rc 0/0 00000100:00100000:3.0:1788845014.628523:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 272 00000100:00100000:0.0:1788845014.631093:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660155136:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845014.631119:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845014.631190:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641350528:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10471us (11683us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845014.631200:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 196 00000100:00100000:3.0:1788845014.653854:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641350912 00000100:00100000:3.0:1788845014.653865:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641350912 00002000:00100000:3.0:1788845014.653950:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1581055) req@ffff94a035445180 x1875739641350912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 504/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.653964:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035445180 with x1875739641350912 ext(1048576->1581055) 00002000:00010000:3.0:1788845014.653970:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035445180 00000100:00100000:3.0:1788845014.654008:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 146 00000100:00100000:3.0:1788845014.654018:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035445180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641350912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.655850:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641350784 00000100:00100000:0.0:1788845014.655863:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641350784 00002000:00100000:0.0:1788845014.655939:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035446680 x1875739641350784/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 504/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.655951:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035446680 with x1875739641350784 00000100:00100000:0.0:1788845014.655977:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 273 00000100:00100000:0.0:1788845014.655986:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035446680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641350784:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845014.668514:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035445180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641350912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14485us (14780us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845014.668530:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 146 00002000:00100000:1.0:1788845014.668544:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1581055) req@ffff94a035445180 x1875739641350912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 504/440 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845014.668555:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035445180 with x1875739641350912 ext(1048576->1581055) 00002000:00010000:1.0:1788845014.668561:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035445180 00000100:00100000:1.0:1788845014.674671:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641350784:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 18676us (21032us total) trans 0 rc 217088/217088 00000100:00100000:1.0:1788845014.674692:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 273 00002000:00100000:1.0:1788845014.674705:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035446680 x1875739641350784/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 504/440 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 217088/217088 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845014.674715:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035446680 with x1875739641350784 00000100:00100000:3.0:1788845014.678698:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641351040 00000100:00100000:3.0:1788845014.678709:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641351040 00002000:00100000:3.0:1788845014.678759:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035446300 x1875739641351040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.678789:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035446300 with x1875739641351040 00000100:00100000:3.0:1788845014.678829:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 274 00000100:00100000:3.0:1788845014.678839:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035446300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641351040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.679100:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641351040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (518us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845014.679108:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 274 00002000:00100000:3.0:1788845014.679115:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035446300 x1875739641351040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.679123:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035446300 with x1875739641351040 00000100:00100000:3.0:1788845014.682852:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641351168 00000100:00100000:3.0:1788845014.682863:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641351168 00002000:00100000:3.0:1788845014.682952:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035445500 x1875739641351168/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.682964:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035445500 with x1875739641351168 00000100:00100000:3.0:1788845014.683006:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 275 00000100:00100000:3.0:1788845014.683016:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641351168:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.683251:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641351168:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 236us (601us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845014.683259:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 275 00002000:00100000:3.0:1788845014.683267:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035445500 x1875739641351168/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.683275:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035445500 with x1875739641351168 00000100:00100000:0.0:1788845014.687152:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.687271:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641351296 00000100:00100000:3.0:1788845014.687287:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641351296 00002000:00100000:3.0:1788845014.687335:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c0e00 x1875739641351296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.687347:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0e00 with x1875739641351296 00000100:00100000:3.0:1788845014.687385:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 276 00000100:00100000:3.0:1788845014.687394:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641351296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.687606:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641351296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 211us (462us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845014.687614:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 276 00002000:00100000:3.0:1788845014.687621:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c0e00 x1875739641351296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.687628:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0e00 with x1875739641351296 00000100:00100000:3.0:1788845014.691049:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641351424 00000100:00100000:3.0:1788845014.691059:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641351424 00002000:00100000:3.0:1788845014.691106:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035447480 x1875739641351424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.691115:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035447480 with x1875739641351424 00000100:00100000:3.0:1788845014.691148:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 277 00000100:00100000:3.0:1788845014.691157:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035447480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641351424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.691390:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035447480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641351424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (679us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845014.691398:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 277 00002000:00100000:3.0:1788845014.691403:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035447480 x1875739641351424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/4536 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.691421:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035447480 with x1875739641351424 00000100:00100000:1.0:1788845014.695141:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.695239:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641351552 00000100:00100000:3.0:1788845014.695249:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641351552 00002000:00100000:3.0:1788845014.695299:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8700 x1875739641351552/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.695311:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8700 with x1875739641351552 00000100:00100000:3.0:1788845014.695351:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 278 00000100:00100000:3.0:1788845014.695360:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641351552:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845014.698539:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 812324 -> 812324 00000004:00010000:3.0:1788845014.698548:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845014 -> 1788845014 00000004:00010000:3.0:1788845014.698550:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845014 -> 1788845014 00000004:00010000:3.0:1788845014.698552:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845014 -> 1788845014 00000004:00010000:3.0:1788845014.698554:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:3.0:1788845014.698624:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641351552:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3260us (3488us total) trans 4294967543 rc 0/0 00000100:00100000:3.0:1788845014.698633:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 278 00002000:00100000:3.0:1788845014.698645:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8700 x1875739641351552/t4294967543(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/448 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.698653:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8700 with x1875739641351552 00000100:00100000:1.0:1788845014.707626:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.707730:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641351680 00000100:00100000:3.0:1788845014.707740:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641351680 00000100:00100000:3.0:1788845014.707800:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:3.0:1788845014.707811:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641351680:12345-192.168.201.29@tcp:16:fsx.0 00000001:00100000:1.0:1788845014.816054:0:6708:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 69859fee-4322-4bed-b50b-7a298318b0f6 00000020:00000080:1.0:1788845014.816064:0:6708:0:(genops.c:885:class_export_put()) final put ffff94a0359d8800/69859fee-4322-4bed-b50b-7a298318b0f6 00000001:00080000:1.0:1788845014.821750:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967543 is committed 00000100:00100000:0.0:1788845014.822008:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641351680:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 114194us (114389us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845014.822020:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:0.0:1788845014.828537:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.831219:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641351808 00000100:00100000:3.0:1788845014.831252:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641351808 00000100:00100000:3.0:1788845014.831384:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 553 00000100:00100000:3.0:1788845014.831396:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641351808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.831420:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.831443:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7428 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.831478:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.831488:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.831525:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.831546:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 14 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.831562:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660155264:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845014.832839:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a004e0f840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.839750:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660155264:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845014.839788:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.839810:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.839814:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845014.839819:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.839831:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01cb0400, rc 0) 00000100:00100000:0.0:1788845014.839926:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641351808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8532us (11397us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845014.839935:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 553 00000100:00100000:1.0:1788845014.840872:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641351936 00000100:00100000:1.0:1788845014.840905:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641351936 00010000:00010000:1.0:1788845014.840940:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 15 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845014.840974:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 197 00000100:00100000:1.0:1788845014.840982:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035446d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641351936:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845014.841006:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845014.841029:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 15 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.841055:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 15 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.841077:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb1200/0xb2187301287ab911 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7421 expref: 15 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.841091:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845014.841096:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.841106:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.841115:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.841149:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845014.841173:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660155392:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845014.841242:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845014.852155:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660155392:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845014.852207:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845014.852266:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641351936:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11284us (12652us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845014.852288:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 197 00000100:00100000:3.0:1788845014.866277:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641352064 00000100:00100000:3.0:1788845014.866287:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641352064 00002000:00100000:3.0:1788845014.866336:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359ac000 x1875739641352064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.866346:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359ac000 with x1875739641352064 00000100:00100000:3.0:1788845014.866385:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 279 00000100:00100000:3.0:1788845014.866409:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641352064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845014.884076:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641352064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17657us (17918us total) trans 0 rc 815104/815104 00000100:00100000:0.0:1788845014.884091:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 279 00002000:00100000:0.0:1788845014.884106:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359ac000 x1875739641352064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:120/0 lens 488/440 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 815104/815104 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845014.884116:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359ac000 with x1875739641352064 00000100:00100000:3.0:1788845014.891433:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845014.891795:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641352192 00000100:00100000:0.0:1788845014.891806:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641352192 00000100:00100000:0.0:1788845014.891875:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 554 00000100:00100000:0.0:1788845014.891936:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641352192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845014.891958:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845014.891978:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab91f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a742f expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.892012:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab91f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a742f expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845014.892023:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 812324 00010000:00010000:0.0:1788845014.892168:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab91f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a742f expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.892185:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.892188:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845014.892192:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d17800, rc 0) 00000100:00100000:1.0:1788845014.892325:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641352192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 390us (900us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845014.892333:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 554 00000100:00100000:3.0:1788845014.912528:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845014.912644:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641352320 00000100:00100000:1.0:1788845014.912654:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641352320 00000100:00100000:1.0:1788845014.912728:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 555 00000100:00100000:1.0:1788845014.912740:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641352320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845014.912762:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845014.912765:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788845014.912809:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d17800/0xb2187301287ab91f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a742f expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.912832:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845014.912838:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845014.912852:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7436 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.912878:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.912945:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.912996:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.913014:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 10 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845014.913040:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660155520:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845014.913123:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845014.917859:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660155520:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845014.917942:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.917964:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845014.917968:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845014.917973:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.917984:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2200, rc 0) 00000100:00100000:1.0:1788845014.918051:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641352320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5313us (5533us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845014.918060:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 555 00000100:00100000:3.0:1788845014.923740:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641352448 00000100:00100000:3.0:1788845014.923750:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641352448 00010000:00010000:3.0:1788845014.923790:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 11 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.923831:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 198 00000100:00100000:3.0:1788845014.923839:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641352448:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845014.923852:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845014.923877:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 11 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.923972:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 11 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.923993:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01cb0400/0xb2187301287ab918 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7428 expref: 11 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.924007:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845014.924013:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.924022:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.924049:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.924089:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.924103:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660155648:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845014.924236:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a004e0f840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845014.925822:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660155648:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845014.925847:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845014.925930:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641352448:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2090us (2283us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845014.925939:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 198 00000100:00100000:3.0:1788845014.931713:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641352576 00000100:00100000:3.0:1788845014.931723:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641352576 00002000:00100000:3.0:1788845014.931770:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359adf80 x1875739641352576/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.931780:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359adf80 with x1875739641352576 00000100:00100000:3.0:1788845014.931816:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 280 00000100:00100000:3.0:1788845014.931825:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641352576:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845014.932121:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641352576:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 295us (513us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845014.932130:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 280 00002000:00100000:3.0:1788845014.932138:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359adf80 x1875739641352576/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/440 e 0 to 0 dl 1788845025 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845014.932145:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359adf80 with x1875739641352576 00000100:00100000:1.0:1788845014.944152:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641352832 00000100:00100000:1.0:1788845014.944172:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641352832 00000100:00100000:1.0:1788845014.944234:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 322 00000100:00100000:1.0:1788845014.944243:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641352832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845014.944504:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845014.944525:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab92d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7444 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845014.944584:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660155776:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845014.944664:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.947499:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641352704 00000100:00100000:3.0:1788845014.947514:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641352704 00000100:00100000:3.0:1788845014.947580:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 556 00000100:00100000:3.0:1788845014.947604:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641352704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.947626:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.947647:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ab934 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a743d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.948823:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.948878:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660155904:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845014.950923:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a004e0f840 going to sleep for 11 seconds 00002000:00010000:1.0:1788845014.956820:0:7983:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1416358 -> 1640152 00002000:00010000:1.0:1788845014.956830:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845013 -> 1788845014 00000100:00100000:1.0:1788845014.956855:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660155776:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845014.956922:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab92d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7444 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.956946:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab92d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7444 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.956959:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2c00, rc 0) 00000100:00100000:1.0:1788845014.957011:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641352832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12769us (13233us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845014.957021:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 322 00000004:00010000:0.0:1788845014.959763:0:5841:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 812324 -> 1048576 00000100:00100000:0.0:1788845014.959793:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660155904:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845014.959818:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845014.959838:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ab934 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a743d expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.959859:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845014.959879:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845014.959923:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ab934 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a743d expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845014.959939:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:0.0:1788845014.960015:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641352704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12413us (16423us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845014.960024:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 556 00000100:00100000:0.0:1788845014.968851:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845014.968945:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641352960 00000100:00100000:3.0:1788845014.968955:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641352960 00000100:00100000:3.0:1788845014.969012:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 557 00000100:00100000:3.0:1788845014.969020:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641352960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845014.969050:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845014.969070:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab93b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a744b expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.969099:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.969108:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab93b lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a744b expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.969142:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.969220:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 14 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845014.969234:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660156032:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845014.969336:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a004e0f840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845014.978414:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660156032:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845014.978451:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab93b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a744b expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.978470:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845014.978473:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845014.978477:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab93b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a744b expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845014.978487:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:3.0:1788845014.978552:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641352960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9532us (9711us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845014.978560:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 557 00000100:00100000:1.0:1788845014.986908:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641353088 00000100:00100000:1.0:1788845014.986918:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641353088 00002000:00100000:1.0:1788845014.986969:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482d80 x1875739641353088/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/0 e 0 to 0 dl 1788845025 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845014.986981:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482d80 with x1875739641353088 00010000:00010000:1.0:1788845014.986991:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 15 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.987026:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 15 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845014.987034:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a010482d80 00000100:00100000:1.0:1788845014.987078:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 14 00000100:00100000:1.0:1788845014.987087:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641353088:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845014.993254:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788845014.993265:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845014 -> 1788845014 00000004:00010000:1.0:1788845014.993268:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845014 -> 1788845014 00000004:00010000:1.0:1788845014.993271:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845014 -> 1788845014 00000004:00010000:1.0:1788845014.993273:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:1.0:1788845014.993361:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641353088:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 6267us (6534us total) trans 4294967544 rc 0/0 00000100:00100000:1.0:1788845014.993373:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 14 00002000:00100000:1.0:1788845014.993388:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482d80 x1875739641353088/t4294967544(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:120/0 lens 488/448 e 0 to 0 dl 1788845025 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845014.993398:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482d80 with x1875739641353088 00010000:00010000:1.0:1788845014.993408:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 17 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845014.993431:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287ab926 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7436 expref: 17 pid: 5839 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845014.993439:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a010482d80 00000100:00100000:3.0:1788845015.000355:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641353216 00000100:00100000:3.0:1788845015.000372:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641353216 00000100:00080000:3.0:1788845015.003147:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845015.011711:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00080000:3.0:1788845015.017207:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845015.031248:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.031404:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641353472 00000100:00100000:0.0:1788845015.031415:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641353472 00000100:00100000:0.0:1788845015.031500:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 323 00000100:00100000:0.0:1788845015.031513:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641353472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845015.031533:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845015.031553:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000072b94c6c/0xb2187301287ab942 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7452 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845015.031624:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660156288:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845015.031717:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0182b3f00 going to sleep for 11 seconds 00000100:00080000:1.0:1788845015.032236:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845015.046633:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.061075:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.071200:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.073140:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641353856 00000100:00100000:1.0:1788845015.073166:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641353856 00000100:00100000:1.0:1788845015.073299:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 326 00000100:00100000:1.0:1788845015.073311:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641353856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.073331:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.073349:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab957 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7467 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.073382:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.073391:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(6 11) 00010000:00010000:1.0:1788845015.073395:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 6) 00010000:00010000:1.0:1788845015.073400:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab957 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1138688->1142783] (req 1138688->1142783) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7467 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.073434:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.073454:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 13 pid: 6685 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.073468:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660156672:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845015.073874:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845015.080420:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.080536:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660156672:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845015.080582:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab957 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1138688->1142783] (req 1138688->1142783) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7467 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.080602:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2c00, rc 0) 00000100:00100000:1.0:1788845015.080668:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641353856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7359us (9479us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845015.080687:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 326 00000100:00100000:3.0:1788845015.081286:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641353984 00000100:00100000:3.0:1788845015.081294:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641353984 00002000:00100000:3.0:1788845015.081357:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1640151) req@ffff94a03f6c3100 x1875739641353984/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 520/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.081367:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c3100 with x1875739641353984 ext(1048576->1640151) 00010000:00010000:3.0:1788845015.081374:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 14 pid: 6685 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.081394:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 14 pid: 6685 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845015.081402:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f6c3100 00000100:00100000:3.0:1788845015.081432:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 6 00000100:00100000:3.0:1788845015.081441:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641353984:12345-192.168.201.29@tcp:4:fsx.0 00000100:00080000:1.0:1788845015.081662:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845015.095421:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641353984:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 13970us (15002us total) trans 4294967355 rc 0/0 00000100:00100000:3.0:1788845015.095437:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 6 00002000:00100000:3.0:1788845015.095449:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1640151) req@ffff94a03f6c3100 x1875739641353984/t4294967355(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 520/456 e 0 to 0 dl 1788845026 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.095459:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c3100 with x1875739641353984 ext(1048576->1640151) 00010000:00010000:3.0:1788845015.095468:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 16 pid: 6685 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.095492:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ab846 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7364 expref: 16 pid: 6685 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845015.095509:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f6c3100 00000100:00100000:0.0:1788845015.098477:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.108634:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010483b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660156800:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845015.108658:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845015.108729:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641354112:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9771us (10262us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845015.108740:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 200 00000100:00100000:1.0:1788845015.109473:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641354240 00000100:00100000:1.0:1788845015.109481:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641354240 00002000:00100000:1.0:1788845015.109521:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1642495) req@ffff94a010480380 x1875739641354240/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.109533:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010480380 with x1875739641354240 ext(1048576->1642495) 00002000:00010000:1.0:1788845015.109538:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010480380 00000100:00100000:1.0:1788845015.109566:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 147 00000100:00100000:1.0:1788845015.109574:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641354240:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845015.118473:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641354240:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8887us (9949us total) trans 0 rc 593920/593920 00000100:00100000:1.0:1788845015.118489:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 147 00002000:00100000:1.0:1788845015.118506:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1642495) req@ffff94a010480380 x1875739641354240/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/440 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 593920/593920 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.118517:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010480380 with x1875739641354240 ext(1048576->1642495) 00002000:00010000:1.0:1788845015.118524:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010480380 00000100:00100000:3.0:1788845015.126871:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.127314:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641354368 00000100:00100000:1.0:1788845015.127341:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641354368 00002000:00100000:1.0:1788845015.127393:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1138688->1150975) req@ffff94a02da49880 x1875739641354368/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 12776/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.127403:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da49880 with x1875739641354368 ext(1138688->1150975) 00002000:00010000:1.0:1788845015.127408:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da49880 00000100:00100000:1.0:1788845015.127447:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 148 00000100:00100000:1.0:1788845015.127458:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641354368:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845015.127946:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641354368:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 488us (1082us total) trans 4294967356 rc 0/0 00000100:00100000:1.0:1788845015.127955:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 148 00002000:00100000:1.0:1788845015.127962:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1138688->1150975) req@ffff94a02da49880 x1875739641354368/t4294967356(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 12776/448 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.127970:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da49880 with x1875739641354368 ext(1138688->1150975) 00002000:00010000:1.0:1788845015.127974:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da49880 00000100:00100000:0.0:1788845015.135311:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000001:00080000:3.0:1788845015.231940:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967354 is committed 00000001:00080000:3.0:1788845015.231975:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967356 is committed 00000001:00080000:3.0:1788845015.232006:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967355 is committed 00000100:00100000:0.0:1788845015.250440:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660157056:0@lo:400:kworker.0 00000100:00100000:0.0:1788845015.250518:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845015.255542:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:3.0:1788845015.255552:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660157184:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845015.255600:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660157184:12345-0@lo:400:kworker.0 Request processed in 50us (1621us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845015.255607:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:3.0:1788845015.255674:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660157184:0@lo:400:kworker.0 00000100:00100000:3.0:1788845015.257916:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.257950:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641354624 00000100:00100000:3.0:1788845015.257959:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641354624 00002000:00100000:3.0:1788845015.258008:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4b100 x1875739641354624/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.258019:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4b100 with x1875739641354624 00000100:00100000:3.0:1788845015.258374:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 282 00000100:00100000:3.0:1788845015.258385:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641354624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00080000:1.0:1788845015.262286:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845015.264480:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660157440:0@lo:400:kworker.0 00000100:00100000:3.0:1788845015.264564:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845015.264614:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788845015.264621:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660157312:0@lo:400:kworker.0 00000100:00100000:3.0:1788845015.264685:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845015.264728:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660157312 00000100:00100000:3.0:1788845015.264735:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660157312 00000100:00100000:3.0:1788845015.264788:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:3.0:1788845015.264796:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660157312:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845015.264850:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660157312:12345-0@lo:400:kworker.0 Request processed in 54us (165us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845015.264858:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00080000:3.0:1788845015.264879:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788845015.264922:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1788845015.264930:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660157568:0@lo:400:kworker.0 00000100:00100000:3.0:1788845015.264955:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845015.264996:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660157312:0@lo:400:kworker.0 00000100:00100000:1.0:1788845015.265385:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660157568 00000100:00100000:1.0:1788845015.265390:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660157568 00000100:00100000:1.0:1788845015.265440:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 114 00000100:00100000:1.0:1788845015.265448:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660157568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845015.265493:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660157568:12345-0@lo:400:kworker.0 Request processed in 46us (536us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845015.265498:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 114 00000100:00100000:0.0:1788845015.266064:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035efad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660157056:0@lo:400:kworker.0 00000100:00080000:3.0:1788845015.269172:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788845015.269182:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845015.269238:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660157568:0@lo:400:kworker.0 00000100:00100000:3.0:1788845015.269281:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660157440:0@lo:400:kworker.0 00000100:00100000:1.0:1788845015.287065:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641354624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 28671us (29146us total) trans 0 rc 815104/815104 00000100:00100000:1.0:1788845015.287082:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 282 00002000:00100000:1.0:1788845015.287111:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4b100 x1875739641354624/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/440 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 815104/815104 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.287122:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4b100 with x1875739641354624 00000100:00100000:3.0:1788845015.297584:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.297635:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.298325:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641354752 00000100:00100000:1.0:1788845015.298330:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641354752 00000100:00100000:1.0:1788845015.298372:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 558 00000100:00100000:1.0:1788845015.298381:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641354752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.298395:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.298410:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2800/0xb2187301287ab965 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a746e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.298437:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2800/0xb2187301287ab965 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a746e expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845015.298450:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845015.298458:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2800/0xb2187301287ab965 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a746e expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.298472:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845015.298475:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845015.298479:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2800, rc 0) 00000100:00100000:1.0:1788845015.298523:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641354752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 143us (887us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845015.298530:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 558 00000100:00100000:0.0:1788845015.310557:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.311177:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641355008 00000100:00100000:1.0:1788845015.311188:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641355008 00000100:00100000:1.0:1788845015.311254:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 329 00000100:00100000:1.0:1788845015.311267:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641355008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.311302:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.311320:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dae64bdf/0xb2187301287ab96c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a747c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.311384:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660157824:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845015.311478:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845015.316459:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660157824:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845015.316503:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000dae64bdf/0xb2187301287ab96c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a747c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.316532:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dae64bdf/0xb2187301287ab96c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a747c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.316546:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3e00, rc 0) 00000100:00100000:1.0:1788845015.316614:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641355008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5350us (6070us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.316624:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 329 00000100:00100000:3.0:1788845015.321244:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.321323:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641355136 00000100:00100000:1.0:1788845015.321334:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641355136 00000100:00100000:1.0:1788845015.321389:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 330 00000100:00100000:1.0:1788845015.321399:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641355136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.321419:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.321438:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287ab973 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7483 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.321524:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660157952:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845015.321618:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845015.327483:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660157952:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845015.327528:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287ab973 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7483 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.327554:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287ab973 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7483 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.327569:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2600, rc 0) 00000100:00100000:1.0:1788845015.327635:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641355136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6236us (6402us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.327645:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 330 00000100:00100000:0.0:1788845015.340367:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.345429:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010480e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660158080:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845015.345484:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033e84272/0xb2187301287ab97a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a748a expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.345517:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033e84272/0xb2187301287ab97a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a748a expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.345532:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023ae800, rc 0) 00000100:00100000:1.0:1788845015.345602:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641355264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5038us (5243us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.345635:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 331 00000100:00100000:0.0:1788845015.351812:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.355961:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.361438:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641355520 00000100:00100000:1.0:1788845015.361457:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641355520 00010000:00010000:1.0:1788845015.361513:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab93b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a744b expref: 11 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.361558:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 201 00000100:00100000:1.0:1788845015.361568:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082fc850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641355520:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845015.361584:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845015.361611:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab93b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a744b expref: 11 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.361630:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab93b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a744b expref: 11 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.361652:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ab93b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a744b expref: 11 pid: 11036 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.361679:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845015.361685:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.361696:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.361704:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.361760:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.361774:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660158336:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845015.362212:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845015.368313:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660158336:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845015.368342:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845015.368404:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082fc850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641355520:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6837us (12447us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845015.368415:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 201 00000100:00100000:3.0:1788845015.369809:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.369939:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641355648 00000100:00100000:1.0:1788845015.369952:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641355648 00002000:00100000:1.0:1788845015.370012:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4b800 x1875739641355648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.370024:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4b800 with x1875739641355648 00000100:00100000:1.0:1788845015.370071:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 283 00000100:00100000:1.0:1788845015.370081:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641355648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845015.370388:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641355648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 304us (581us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845015.370572:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 283 00002000:00100000:1.0:1788845015.370581:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4b800 x1875739641355648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.370592:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4b800 with x1875739641355648 00000100:00100000:0.0:1788845015.379931:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.380201:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641355776 00000100:00100000:3.0:1788845015.380212:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641355776 00002000:00100000:3.0:1788845015.380265:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c3b80 x1875739641355776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.380299:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c3b80 with x1875739641355776 00000100:00100000:3.0:1788845015.380329:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 284 00000100:00100000:3.0:1788845015.380339:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641355776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845015.380618:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641355776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 279us (735us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845015.380626:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 284 00002000:00100000:3.0:1788845015.380634:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c3b80 x1875739641355776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.380643:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c3b80 with x1875739641355776 00000100:00100000:3.0:1788845015.390423:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.390522:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641355904 00000100:00100000:1.0:1788845015.390534:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641355904 00000100:00100000:1.0:1788845015.390597:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 560 00000100:00100000:1.0:1788845015.390607:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641355904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.390627:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.390648:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2600/0xb2187301287ab988 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7498 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.390698:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.390740:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660158464:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845015.390830:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895900 going to sleep for 11 seconds 00000004:00010000:1.0:1788845015.393493:0:5840:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845014 -> 1788845015 00000004:00010000:1.0:1788845015.393504:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845014 -> 1788845015 00000004:00010000:1.0:1788845015.393507:0:5840:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845014 -> 1788845015 00000100:00100000:1.0:1788845015.393534:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660158464:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845015.393560:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845015.393578:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2600/0xb2187301287ab988 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7498 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.393600:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845015.393603:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845015.393609:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2600/0xb2187301287ab988 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7498 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.393630:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2600, rc 0) 00000100:00100000:1.0:1788845015.393705:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641355904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3099us (3291us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.393715:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 560 00000100:00100000:0.0:1788845015.397416:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.397493:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641356032 00000100:00100000:1.0:1788845015.397504:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641356032 00000100:00100000:1.0:1788845015.397572:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 561 00000100:00100000:1.0:1788845015.397583:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641356032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.397604:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.397624:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3e00/0xb2187301287ab98f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a749f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.397660:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.397701:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660158592:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845015.397785:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845015.400490:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660158592:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845015.400520:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845015.400539:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3e00/0xb2187301287ab98f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a749f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.400559:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845015.400561:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845015.400566:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3e00/0xb2187301287ab98f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a749f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.400580:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3e00, rc 0) 00000100:00100000:1.0:1788845015.400660:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641356032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3078us (3254us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.400668:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 561 00000100:00100000:0.0:1788845015.407794:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.416622:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.417171:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641356416 00000100:00100000:3.0:1788845015.417182:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641356416 00010000:00010000:3.0:1788845015.417231:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af400/0xb2187301287ab854 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a736b expref: 10 pid: 5841 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.417275:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 202 00000100:00100000:3.0:1788845015.417285:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641356416:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845015.417301:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845015.417308:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af400/0xb2187301287ab854 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a736b expref: 10 pid: 5841 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.417326:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af400/0xb2187301287ab854 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a736b expref: 10 pid: 5841 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.417348:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845015.417372:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.417383:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.417390:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.417412:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845015.417465:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641356416:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 181us (853us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845015.417474:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 202 00010000:00010000:3.0:1788845015.429915:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.429932:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.429941:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.429947:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f023af600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845015.429952:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.429961:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.429979:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845015.429982:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845015.429986:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.430009:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af600/0xb2187301287ab996 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845015.431644:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.431917:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641356544 00000100:00100000:3.0:1788845015.431926:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641356544 00000100:00100000:3.0:1788845015.431987:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 332 00000100:00100000:3.0:1788845015.431997:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641356544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845015.432015:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845015.432274:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a74a6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845015.432311:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab957 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1138688->1142783) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7467 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845015.432322:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 6) 00010000:00010000:0.0:1788845015.432326:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 6) 00010000:00010000:0.0:1788845015.432331:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74a6 expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845015.432369:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab957 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1138688->1142783) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7467 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845015.432406:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000df839bec/0xb2187301287ab957 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1138688->1142783) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7467 expref: 5 pid: 7983 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845015.432419:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660158848:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845015.432502:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845015.439349:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.448200:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.448589:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641356800 00000100:00100000:1.0:1788845015.448596:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641356800 00002000:00010000:1.0:1788845015.448632:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1691448->18446744073709551615) 00002000:00100000:1.0:1788845015.448637:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da4bb80 with x1875739641356800 ext(1691448->18446744073709551615) 00002000:00010000:1.0:1788845015.448647:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da4bb80. 00000100:00100000:1.0:1788845015.448682:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 149 00000100:00100000:1.0:1788845015.448690:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641356800:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845015.449008:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1640152 -> 1691448 00002000:00010000:1.0:1788845015.449012:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845015 -> 1788845015 00002000:00010000:1.0:1788845015.449015:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845015 -> 1788845014 00002000:00010000:1.0:1788845015.449018:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845015 -> 1788845015 00002000:00010000:1.0:1788845015.449020:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 259 -> 203 00000100:00100000:1.0:1788845015.449061:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641356800:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 371us (873us total) trans 4294967357 rc 0/0 00000100:00100000:1.0:1788845015.449068:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 149 00002000:00010000:1.0:1788845015.449071:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1691448->18446744073709551615) 00002000:00100000:1.0:1788845015.449074:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da4bb80 with x1875739641356800 ext(1691448->18446744073709551615) 00002000:00010000:1.0:1788845015.449078:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da4bb80. 00000100:00100000:0.0:1788845015.454353:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.473088:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.473136:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.474002:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641357184 00000100:00100000:1.0:1788845015.474008:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641357184 00002000:00100000:1.0:1788845015.474050:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1691647) req@ffff949f0251f480 x1875739641357184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.474061:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251f480 with x1875739641357184 ext(1048576->1691647) 00002000:00010000:1.0:1788845015.474080:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251f480 00000100:00100000:1.0:1788845015.474110:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 150 00000100:00100000:1.0:1788845015.474118:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641357184:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845015.487682:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641357184:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 13552us (14533us total) trans 0 rc 643072/643072 00000100:00100000:1.0:1788845015.487698:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 150 00002000:00100000:1.0:1788845015.487712:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1691647) req@ffff949f0251f480 x1875739641357184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 488/440 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 643072/643072 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.487723:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251f480 with x1875739641357184 ext(1048576->1691647) 00002000:00010000:1.0:1788845015.487729:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251f480 00000100:00100000:3.0:1788845015.512686:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.520258:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641357312 00000100:00100000:1.0:1788845015.520274:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641357312 00000100:00100000:0.0:1788845015.528262:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.537628:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00002000:00010000:1.0:1788845015.546789:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845014 -> 1788845015 00000100:00100000:1.0:1788845015.546823:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010483b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660159232:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845015.546862:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c5ce56c9/0xb2187301287ab9c7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74c2 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.546925:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c5ce56c9/0xb2187301287ab9c7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74c2 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.546938:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023af200, rc 0) 00000100:00100000:1.0:1788845015.546998:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641357568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8941us (9381us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.547040:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 333 00000100:00100000:3.0:1788845015.556557:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.556596:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.557071:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641357824 00000100:00100000:1.0:1788845015.557076:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641357824 00000100:00100000:1.0:1788845015.557108:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 334 00000100:00100000:1.0:1788845015.557116:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641357824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.557125:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.557137:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287ab9d5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a74d0 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.557321:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660159488:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845015.557352:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845015.560665:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660159488:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845015.560710:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287ab9d5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74d0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.560741:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287ab9d5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74d0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.560759:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2600, rc 0) 00000100:00100000:1.0:1788845015.560835:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641357824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3720us (4288us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.560846:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 334 00000100:00100000:3.0:1788845015.566401:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.566445:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.566527:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641357952 00000100:00100000:1.0:1788845015.566539:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641357952 00000100:00100000:1.0:1788845015.566620:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 567 00000100:00100000:1.0:1788845015.566630:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641357952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.566654:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.566686:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2600/0xb2187301287ab9dc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a74d7 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.566726:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.566770:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660159616:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845015.566857:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845015.571824:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660159616:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845015.571855:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845015.571875:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2600/0xb2187301287ab9dc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74d7 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.571926:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845015.571929:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845015.571934:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2600/0xb2187301287ab9dc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74d7 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.571948:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2600, rc 0) 00000100:00100000:1.0:1788845015.572008:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641357952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5379us (5617us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.572017:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 567 00000100:00100000:0.0:1788845015.578513:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.578602:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.578789:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641358208 00000100:00100000:1.0:1788845015.578802:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641358208 00000100:00100000:1.0:1788845015.578919:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 568 00000100:00100000:1.0:1788845015.578930:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641358208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.578952:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.578972:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2600/0xb2187301287ab9ea lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a74e5 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.579012:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.579070:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660159872:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845015.579163:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845015.592839:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.593660:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641358464 00000100:00100000:3.0:1788845015.593682:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641358464 00000100:00100000:3.0:1788845015.593743:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 569 00000100:00100000:3.0:1788845015.593752:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641358464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845015.593773:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845015.593793:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ab9f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a74f3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.593830:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.593870:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660160128:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845015.594388:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845015.607278:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.607412:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641358592 00000100:00100000:1.0:1788845015.607422:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641358592 00000100:00100000:1.0:1788845015.607486:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 337 00000100:00100000:1.0:1788845015.607497:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641358592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.607516:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.607534:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000073457540/0xb2187301287ab9ff lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a74fa expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.607594:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660160256:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845015.607774:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660160128:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845015.607796:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845015.607813:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ab9f8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74f3 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.607832:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845015.607836:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845015.607841:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ab9f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74f3 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.607856:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845015.607970:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845015.618121:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641358464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 24358us (25279us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845015.618136:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 569 00000100:00100000:1.0:1788845015.626935:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660160256:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845015.626978:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000073457540/0xb2187301287ab9ff lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74fa expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.627006:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000073457540/0xb2187301287ab9ff lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74fa expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.627036:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2200, rc 0) 00000100:00100000:1.0:1788845015.627094:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641358592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 19598us (19828us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.627104:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 337 00000100:00100000:0.0:1788845015.630484:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.630552:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641358720 00000100:00100000:1.0:1788845015.630564:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641358720 00000100:00100000:1.0:1788845015.630622:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 338 00000100:00100000:1.0:1788845015.630632:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641358720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.630650:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.630675:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000415b81e5/0xb2187301287aba06 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7501 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.630707:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74a6 expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.630716:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 6) 00010000:00010000:1.0:1788845015.630720:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 6) 00010000:00010000:1.0:1788845015.630726:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000415b81e5/0xb2187301287aba06 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1376256->1380351] (req 1376256->1380351) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7501 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.630757:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74a6 expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.630775:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74a6 expref: 6 pid: 6684 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.630802:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660160384:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845015.630879:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008895900 going to sleep for 11 seconds 00010000:00010000:1.0:1788845015.634387:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845015.638331:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.638410:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641358848 00000100:00100000:3.0:1788845015.638420:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641358848 00010000:00010000:3.0:1788845015.638468:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74a6 expref: 7 pid: 6684 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.638512:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 204 00000100:00100000:3.0:1788845015.638523:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641358848:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845015.638546:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845015.638572:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74a6 expref: 7 pid: 6684 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.638609:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74a6 expref: 7 pid: 6684 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.638637:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 6) 00010000:00010000:3.0:1788845015.638641:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 6) 00010000:00010000:3.0:1788845015.638651:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000415b81e5/0xb2187301287aba06 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7501 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.638661:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000415b81e5/0xb2187301287aba06 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7501 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.638702:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000415b81e5/0xb2187301287aba06 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7501 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.638718:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660160512:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845015.638818:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b3c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788845015.639326:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287ab99d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74a6 expref: 7 pid: 6684 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.639343:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660160384:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845015.639369:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000415b81e5/0xb2187301287aba06 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7501 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.639379:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3c00, rc 0) 00000100:00100000:1.0:1788845015.639422:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641358720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8790us (8948us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845015.639430:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 338 00000100:00100000:1.0:1788845015.644987:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660160512:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845015.645017:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845015.645087:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641358848:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6564us (6766us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845015.645099:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 204 00000100:00100000:3.0:1788845015.647935:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.658041:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.662605:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641359104 00000100:00100000:1.0:1788845015.662622:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641359104 00002000:00100000:1.0:1788845015.663247:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1380352->1384447) req@ffff949f0251c700 x1875739641359104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.663277:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251c700 with x1875739641359104 ext(1380352->1384447) 00002000:00010000:1.0:1788845015.663283:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251c700 00000100:00100000:1.0:1788845015.663313:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 152 00000100:00100000:1.0:1788845015.663323:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845015.663566:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 245us (5547us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845015.663573:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 152 00002000:00100000:1.0:1788845015.663578:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1380352->1384447) req@ffff949f0251c700 x1875739641359104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.663585:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251c700 with x1875739641359104 ext(1380352->1384447) 00002000:00010000:1.0:1788845015.663589:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251c700 00000100:00100000:0.0:1788845015.666937:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.667095:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641359232 00000100:00100000:1.0:1788845015.667108:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641359232 00002000:00100000:1.0:1788845015.667173:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1384448->1388543) req@ffff949f0251e300 x1875739641359232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.667185:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251e300 with x1875739641359232 ext(1384448->1388543) 00002000:00010000:1.0:1788845015.667191:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251e300 00000100:00100000:1.0:1788845015.667221:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 153 00000100:00100000:1.0:1788845015.667230:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359232:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845015.667453:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 223us (525us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845015.667460:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 153 00002000:00100000:1.0:1788845015.667481:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1384448->1388543) req@ffff949f0251e300 x1875739641359232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.667490:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251e300 with x1875739641359232 ext(1384448->1388543) 00002000:00010000:1.0:1788845015.667493:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251e300 00000100:00100000:0.0:1788845015.671789:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.673769:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641359360 00000100:00100000:3.0:1788845015.673783:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641359360 00002000:00100000:3.0:1788845015.673843:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1388544->1392639) req@ffff949f036cce00 x1875739641359360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.673854:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f036cce00 with x1875739641359360 ext(1388544->1392639) 00002000:00010000:3.0:1788845015.673859:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f036cce00 00000100:00100000:3.0:1788845015.674179:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 154 00000100:00100000:3.0:1788845015.674205:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845015.674438:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359360:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 235us (2659us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845015.674445:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 154 00002000:00100000:3.0:1788845015.674451:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1388544->1392639) req@ffff949f036cce00 x1875739641359360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.674459:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f036cce00 with x1875739641359360 ext(1388544->1392639) 00002000:00010000:3.0:1788845015.674462:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f036cce00 00000100:00100000:0.0:1788845015.677956:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.682009:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.682173:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641359616 00000100:00100000:3.0:1788845015.682185:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641359616 00002000:00100000:3.0:1788845015.682234:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1396736->1400831) req@ffff949f036cf100 x1875739641359616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.682246:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f036cf100 with x1875739641359616 ext(1396736->1400831) 00002000:00010000:3.0:1788845015.682251:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f036cf100 00000100:00100000:3.0:1788845015.682276:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 156 00000100:00100000:3.0:1788845015.682286:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845015.682516:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 230us (517us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845015.682524:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 156 00002000:00100000:3.0:1788845015.682531:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1396736->1400831) req@ffff949f036cf100 x1875739641359616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.682538:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f036cf100 with x1875739641359616 ext(1396736->1400831) 00002000:00010000:3.0:1788845015.682541:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f036cf100 00000100:00100000:0.0:1788845015.686710:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.686927:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641359744 00000100:00100000:3.0:1788845015.686937:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641359744 00002000:00100000:3.0:1788845015.686990:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1400832->1404927) req@ffff94a03d5dd880 x1875739641359744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.687002:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5dd880 with x1875739641359744 ext(1400832->1404927) 00002000:00010000:3.0:1788845015.687009:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5dd880 00000100:00100000:3.0:1788845015.687056:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 157 00000100:00100000:3.0:1788845015.687069:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845015.687292:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641359744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 226us (591us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845015.687300:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 157 00002000:00100000:3.0:1788845015.687307:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1400832->1404927) req@ffff94a03d5dd880 x1875739641359744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.687315:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5dd880 with x1875739641359744 ext(1400832->1404927) 00002000:00010000:3.0:1788845015.687318:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5dd880 00000100:00100000:0.0:1788845015.690520:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.694312:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.694398:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641360000 00000100:00100000:1.0:1788845015.694408:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641360000 00002000:00100000:1.0:1788845015.694451:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1409024->1413119) req@ffff94a0345d1880 x1875739641360000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.694463:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0345d1880 with x1875739641360000 ext(1409024->1413119) 00002000:00010000:1.0:1788845015.694468:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0345d1880 00000100:00100000:1.0:1788845015.694499:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 159 00000100:00100000:1.0:1788845015.694508:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641360000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845015.694702:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641360000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 194us (399us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845015.694709:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 159 00002000:00100000:1.0:1788845015.694716:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1409024->1413119) req@ffff94a0345d1880 x1875739641360000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.694724:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0345d1880 with x1875739641360000 ext(1409024->1413119) 00002000:00010000:1.0:1788845015.694727:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0345d1880 00000100:00100000:0.0:1788845015.699201:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.699297:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641360128 00000100:00100000:3.0:1788845015.699307:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641360128 00002000:00100000:3.0:1788845015.699361:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1413120->1417215) req@ffff949f0358dc00 x1875739641360128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.699372:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358dc00 with x1875739641360128 ext(1413120->1417215) 00002000:00010000:3.0:1788845015.699378:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358dc00 00000100:00100000:3.0:1788845015.699416:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 160 00000100:00100000:3.0:1788845015.699427:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641360128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845015.699641:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641360128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 215us (452us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845015.699664:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 160 00002000:00100000:3.0:1788845015.699684:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1413120->1417215) req@ffff949f0358dc00 x1875739641360128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.699693:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358dc00 with x1875739641360128 ext(1413120->1417215) 00002000:00010000:3.0:1788845015.699697:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358dc00 00000100:00100000:0.0:1788845015.728083:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.754256:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.754345:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641360384 00000100:00100000:3.0:1788845015.754356:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641360384 00000100:00100000:3.0:1788845015.754421:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 570 00000100:00100000:3.0:1788845015.754432:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641360384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845015.754456:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845015.754478:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba14 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a750f expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.754516:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.754557:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035928700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660160640:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845015.754646:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845015.756695:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035928700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660160640:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845015.756723:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845015.756742:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba14 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a750f expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.756765:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845015.756768:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845015.756790:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba14 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a750f expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.756809:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845015.756870:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641360384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2440us (2624us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845015.756879:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 570 00000100:00100000:0.0:1788845015.759828:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.763261:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660161024:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845015.763341:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845015.767583:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010480e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660160768:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845015.767603:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010483b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660160896:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845015.767642:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a80c1dd2/0xb2187301287aba1b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1363968->1368063] (req 1363968->1368063) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7516 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.767677:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023af800, rc 0) 00000100:00100000:3.0:1788845015.767745:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641360512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7715us (7926us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845015.767754:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 340 00000100:00100000:3.0:1788845015.772861:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660161024:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845015.774270:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.774315:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.779431:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.786243:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.788472:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641361024 00000100:00100000:1.0:1788845015.788485:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641361024 00002000:00100000:1.0:1788845015.788547:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1368064->1372159) req@ffff94a0360b5500 x1875739641361024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.788557:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5500 with x1875739641361024 ext(1368064->1372159) 00002000:00010000:1.0:1788845015.788575:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5500 00000100:00100000:1.0:1788845015.788609:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 162 00000100:00100000:1.0:1788845015.788616:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641361024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845015.789055:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641361024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 435us (2820us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845015.789064:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 162 00002000:00100000:1.0:1788845015.789071:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1368064->1372159) req@ffff94a0360b5500 x1875739641361024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.789080:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5500 with x1875739641361024 ext(1368064->1372159) 00002000:00010000:1.0:1788845015.789084:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5500 00000100:00100000:0.0:1788845015.802576:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.802661:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641361152 00000100:00100000:3.0:1788845015.802670:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641361152 00002000:00100000:3.0:1788845015.802718:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1372160->1376255) req@ffff94a0360b7800 x1875739641361152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.802730:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7800 with x1875739641361152 ext(1372160->1376255) 00002000:00010000:3.0:1788845015.802735:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7800 00000100:00100000:3.0:1788845015.802780:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 163 00000100:00100000:3.0:1788845015.802788:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641361152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845015.803330:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641361152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 539us (762us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845015.803339:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 163 00002000:00100000:3.0:1788845015.803346:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1372160->1376255) req@ffff94a0360b7800 x1875739641361152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.803354:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7800 with x1875739641361152 ext(1372160->1376255) 00002000:00010000:3.0:1788845015.803357:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7800 00000100:00100000:0.0:1788845015.820162:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.829355:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.829770:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641361408 00000100:00100000:1.0:1788845015.829781:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641361408 00002000:00100000:1.0:1788845015.829833:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1380352->1384447) req@ffff94a03f655c00 x1875739641361408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.829843:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f655c00 with x1875739641361408 ext(1380352->1384447) 00002000:00010000:1.0:1788845015.829848:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f655c00 00000100:00100000:1.0:1788845015.829912:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 165 00000100:00100000:1.0:1788845015.829923:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641361408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845015.830563:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641361408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 639us (1216us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845015.830572:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 165 00002000:00100000:1.0:1788845015.830579:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1380352->1384447) req@ffff94a03f655c00 x1875739641361408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/4536 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.830588:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f655c00 with x1875739641361408 ext(1380352->1384447) 00002000:00010000:1.0:1788845015.830598:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f655c00 00000100:00100000:0.0:1788845015.839967:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.840063:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641361536 00000100:00100000:3.0:1788845015.840073:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641361536 00002000:00100000:3.0:1788845015.840124:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1363968->1384447) req@ffff94a03f654e00 x1875739641361536/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.840136:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f654e00 with x1875739641361536 ext(1363968->1384447) 00002000:00010000:3.0:1788845015.840152:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f654e00 00000100:00100000:3.0:1788845015.840189:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 166 00000100:00100000:3.0:1788845015.840200:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641361536:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845015.863724:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641361536:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 23514us (23756us total) trans 4294967358 rc 0/0 00000100:00100000:1.0:1788845015.863739:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 166 00002000:00100000:1.0:1788845015.863751:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1363968->1384447) req@ffff94a03f654e00 x1875739641361536/t4294967358(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:121/0 lens 488/448 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845015.863761:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f654e00 with x1875739641361536 ext(1363968->1384447) 00002000:00010000:1.0:1788845015.863767:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f654e00 00000100:00100000:0.0:1788845015.867698:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000001:00080000:3.0:1788845015.911033:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967358 is committed 00000001:00080000:3.0:1788845015.911047:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967357 is committed 00000100:00100000:3.0:1788845015.914617:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.914949:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641361792 00000100:00100000:1.0:1788845015.914959:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641361792 00000100:00100000:1.0:1788845015.915077:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 571 00000100:00100000:1.0:1788845015.915089:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641361792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845015.915111:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845015.915131:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287aba22 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a751d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.915178:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.915216:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660161280:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845015.915379:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003147b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845015.917507:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660161280:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845015.917532:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845015.917550:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287aba22 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a751d expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.917591:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845015.917593:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845015.917598:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287aba22 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a751d expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.917612:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2400, rc 0) 00000100:00100000:3.0:1788845015.917798:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641361792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2710us (3189us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845015.917807:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 571 00000100:00100000:0.0:1788845015.920508:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.925910:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010481880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660161408:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845015.925942:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845015.925961:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afa00/0xb2187301287aba29 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7524 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.926006:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845015.926008:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845015.926189:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023afa00/0xb2187301287aba29 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7524 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.926206:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f023afa00, rc 0) 00000100:00100000:1.0:1788845015.926272:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641361920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5467us (5773us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845015.926280:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 572 00000100:00100000:1.0:1788845015.929409:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.929487:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641362048 00000100:00100000:3.0:1788845015.929496:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641362048 00000100:00100000:3.0:1788845015.929544:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 342 00000100:00100000:3.0:1788845015.929552:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641362048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845015.929570:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845015.929587:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba30 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a752b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.929642:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660161536:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845015.929735:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00002000:00010000:3.0:1788845015.931926:0:7983:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 203 -> 243 00000100:00100000:3.0:1788845015.931956:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660161536:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845015.931996:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba30 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a752b expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.932053:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba30 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a752b expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.932066:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845015.932125:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641362048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2572us (2723us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845015.932134:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 342 00000100:00100000:1.0:1788845015.935590:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845015.943302:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.943393:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641362304 00000100:00100000:3.0:1788845015.943404:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641362304 00000100:00100000:3.0:1788845015.943460:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 344 00000100:00100000:3.0:1788845015.943469:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641362304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845015.943487:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845015.943504:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba3e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7539 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.943556:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660161792:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845015.943636:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845015.945766:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660161792:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845015.945809:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba3e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7539 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.945835:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba3e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7539 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.945870:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845015.946015:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641362304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2546us (2722us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845015.946033:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 344 00000100:00100000:0.0:1788845015.949856:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.950216:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641362432 00000100:00100000:3.0:1788845015.950228:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641362432 00000100:00100000:3.0:1788845015.950304:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 345 00000100:00100000:3.0:1788845015.950316:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641362432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845015.950336:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845015.950355:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7540 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.950390:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a80c1dd2/0xb2187301287aba1b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1363968->1368063) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7516 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.950400:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(3 6) 00010000:00010000:3.0:1788845015.950404:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 6) 00010000:00010000:3.0:1788845015.950409:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1052671] (req 1048576->1052671) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.950444:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a80c1dd2/0xb2187301287aba1b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1363968->1368063) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7516 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.950463:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a80c1dd2/0xb2187301287aba1b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1363968->1368063) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7516 expref: 5 pid: 6684 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.950494:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660161920:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845015.950570:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845015.953845:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.953935:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641362560 00000100:00100000:1.0:1788845015.953944:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641362560 00010000:00010000:1.0:1788845015.953985:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a80c1dd2/0xb2187301287aba1b lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1363968->1368063) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7516 expref: 6 pid: 6684 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.954030:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 207 00000100:00100000:1.0:1788845015.954039:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641362560:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845015.954053:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845015.954075:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a80c1dd2/0xb2187301287aba1b lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1363968->1368063) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7516 expref: 6 pid: 6684 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.954091:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a80c1dd2/0xb2187301287aba1b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1363968->1368063) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7516 expref: 6 pid: 6684 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.954134:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 6) 00010000:00010000:1.0:1788845015.954148:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 6) 00010000:00010000:1.0:1788845015.954156:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.954166:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845015.954202:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845015.954217:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660162048:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845015.954639:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a80c1dd2/0xb2187301287aba1b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1363968->1368063) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7516 expref: 6 pid: 6684 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845015.954666:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660161920:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845015.954691:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845015.954700:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845015.954805:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0031473c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845015.955548:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641362432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5230us (5698us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845015.955557:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 345 00000100:00100000:0.0:1788845015.962823:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845015.964808:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660162048:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845015.964839:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845015.964924:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641362560:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10884us (11087us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845015.964933:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 207 00000100:00100000:0.0:1788845015.999106:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845015.999281:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641362816 00000100:00100000:3.0:1788845015.999291:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641362816 00002000:00100000:3.0:1788845015.999340:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4b480 x1875739641362816/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 504/0 e 0 to 0 dl 1788845026 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845015.999350:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4b480 with x1875739641362816 00000100:00100000:3.0:1788845015.999387:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 286 00000100:00100000:3.0:1788845015.999396:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641362816:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845016.001694:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.001860:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641362944 00000100:00100000:3.0:1788845016.001870:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641362944 00002000:00100000:3.0:1788845016.001945:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1101823) req@ffff94a035ef9500 x1875739641362944/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.001964:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035ef9500 with x1875739641362944 ext(1048576->1101823) 00002000:00010000:3.0:1788845016.001971:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035ef9500 00000100:00100000:3.0:1788845016.002026:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 168 00000100:00100000:3.0:1788845016.002037:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641362944:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845016.004723:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845016.004732:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845015 -> 1788845016 00000004:00010000:3.0:1788845016.004735:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845015 -> 1788845014 00000004:00010000:3.0:1788845016.004737:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845015 -> 1788845016 00000004:00010000:3.0:1788845016.004738:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:3.0:1788845016.004824:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641362816:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5424us (5722us total) trans 4294967547 rc 0/0 00000100:00100000:3.0:1788845016.004834:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 286 00002000:00100000:3.0:1788845016.004846:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4b480 x1875739641362816/t4294967547(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:121/0 lens 504/448 e 0 to 0 dl 1788845026 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.004855:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4b480 with x1875739641362816 00000100:00100000:3.0:1788845016.009739:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641362944:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 7694us (8044us total) trans 4294967359 rc 0/0 00000100:00100000:3.0:1788845016.009752:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 168 00002000:00100000:3.0:1788845016.009767:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1101823) req@ffff94a035ef9500 x1875739641362944/t4294967359(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/448 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.009777:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035ef9500 with x1875739641362944 ext(1048576->1101823) 00002000:00010000:3.0:1788845016.009783:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035ef9500 00000100:00100000:0.0:1788845016.013113:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.013171:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.013330:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641363200 00000100:00100000:1.0:1788845016.013340:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641363200 00000100:00100000:1.0:1788845016.013401:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 346 00000100:00100000:1.0:1788845016.013412:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641363200:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:3.0:1788845016.013533:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641363072 00000100:00100000:3.0:1788845016.013538:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641363072 00000100:00100000:3.0:1788845016.013578:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 573 00000100:00100000:3.0:1788845016.013586:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641363072:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:1.0:1788845016.044944:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641363072:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 31346us (31827us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845016.044971:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 573 00000001:00080000:1.0:1788845016.045029:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967544 is committed 00000001:00080000:1.0:1788845016.045043:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967546 is committed 00000001:00080000:1.0:1788845016.045050:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967547 is committed 00000100:00100000:3.0:1788845016.054593:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641363200:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 41178us (41416us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845016.054604:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 346 00000100:00100000:0.0:1788845016.058107:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.058156:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.058751:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641363328 00000100:00100000:3.0:1788845016.058756:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641363328 00000100:00100000:3.0:1788845016.058801:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 574 00000100:00100000:3.0:1788845016.058809:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641363328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845016.058820:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845016.058834:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba53 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7547 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.058861:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.058911:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660162304:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845016.058947:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b900 going to sleep for 11 seconds 00000004:00010000:3.0:1788845016.063593:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845014 -> 1788845015 00000004:00010000:3.0:1788845016.063602:0:5840:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 390 -> 584 00000100:00100000:3.0:1788845016.063629:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660162304:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845016.063736:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845016.063757:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba53 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7547 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.063791:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845016.063795:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845016.063799:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7547 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.063808:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845016.064225:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641363328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5415us (6125us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845016.064234:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 574 00000100:00100000:0.0:1788845016.077432:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641363712 00000100:00100000:0.0:1788845016.077445:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641363712 00000100:00100000:0.0:1788845016.077511:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 348 00000100:00100000:0.0:1788845016.077523:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641363712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.077547:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.077570:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287aba5a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a755c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.077727:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660162432:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845016.077819:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.077921:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641363584 00000100:00100000:1.0:1788845016.077925:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641363584 00000100:00100000:1.0:1788845016.078111:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 575 00000100:00100000:1.0:1788845016.078120:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641363584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.078132:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.078156:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2e00/0xb2187301287aba61 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7555 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.078187:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.078213:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660162560:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845016.078248:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a012d330c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845016.087123:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660162432:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845016.087193:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287aba5a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a755c expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.087217:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287aba5a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a755c expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.087229:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b200, rc 0) 00000100:00100000:0.0:1788845016.087288:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641363712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9768us (9896us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845016.087297:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 348 00000100:00100000:1.0:1788845016.087404:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660162560:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845016.087423:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845016.087435:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2e00/0xb2187301287aba61 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7555 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.087446:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845016.087448:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845016.087452:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2e00/0xb2187301287aba61 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7555 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.087479:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2e00, rc 0) 00000100:00100000:1.0:1788845016.087500:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641363584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9383us (10169us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845016.087506:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 575 00000100:00100000:0.0:1788845016.106362:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641363968 00000100:00100000:0.0:1788845016.106372:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641363968 00000100:00100000:0.0:1788845016.106433:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 349 00000100:00100000:0.0:1788845016.106445:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035446d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641363968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.106465:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.106484:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287aba68 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a756a expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.106547:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660162688:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845016.106633:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.108150:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641363840 00000100:00100000:1.0:1788845016.108162:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641363840 00000100:00100000:0.0:1788845016.110503:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660162688:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845016.110547:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287aba68 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a756a expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.110573:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287aba68 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a756a expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.110588:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b200, rc 0) 00000100:00100000:0.0:1788845016.110665:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641363968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4221us (4500us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845016.110674:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 349 00000100:00100000:3.0:1788845016.130511:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.133454:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641364096 00000100:00100000:1.0:1788845016.133472:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641364096 00000100:00100000:1.0:1788845016.133552:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 577 00000100:00100000:1.0:1788845016.133561:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641364096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.133589:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.133611:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7571 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.133641:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.133651:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.133684:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.133699:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 14 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.133729:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660162944:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845016.133822:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a012d330c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.139196:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.139294:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641364224 00000100:00100000:0.0:1788845016.139304:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641364224 00010000:00010000:0.0:1788845016.139341:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 15 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.139393:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 208 00000100:00100000:0.0:1788845016.139402:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082fe850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641364224:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845016.139417:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845016.139444:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 15 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.139461:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 15 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.139484:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845016.139505:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.139515:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.139524:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.139559:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.139572:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660163072:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845016.139662:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f540 going to sleep for 11 seconds 00010000:00010000:3.0:1788845016.140186:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aea00/0xb2187301287ab981 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7491 expref: 15 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.140203:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660162944:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845016.140226:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.140237:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845016.140240:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845016.140242:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3800, rc 0) 00000100:00100000:3.0:1788845016.140317:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641364096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6757us (9816us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845016.140325:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 577 00000100:00100000:0.0:1788845016.156474:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660163072:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845016.156503:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845016.156554:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082fe850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641364224:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 17152us (17368us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845016.156566:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 208 00000100:00100000:1.0:1788845016.167522:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.167739:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641364352 00000100:00100000:0.0:1788845016.167750:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641364352 00002000:00100000:0.0:1788845016.167803:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8a80 x1875739641364352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845016.167814:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8a80 with x1875739641364352 00000100:00100000:0.0:1788845016.167860:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 287 00000100:00100000:0.0:1788845016.167870:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641364352:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845016.253642:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641364352:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 85763us (86120us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1788845016.253655:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 287 00002000:00100000:0.0:1788845016.253668:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8a80 x1875739641364352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:122/0 lens 488/440 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845016.253692:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8a80 with x1875739641364352 00000100:00100000:0.0:1788845016.265032:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.265142:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641364480 00000100:00100000:1.0:1788845016.265152:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641364480 00002000:00100000:1.0:1788845016.265205:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c1f80 x1875739641364480/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.265217:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c1f80 with x1875739641364480 00000100:00100000:1.0:1788845016.265249:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 288 00000100:00100000:1.0:1788845016.265258:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641364480:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845016.277287:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788845016.277297:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845016 -> 1788845016 00000004:00010000:1.0:1788845016.277300:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845015 -> 1788845014 00000004:00010000:1.0:1788845016.277302:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845016 -> 1788845016 00000004:00010000:1.0:1788845016.277304:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:1.0:1788845016.277403:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641364480:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 12140us (12388us total) trans 4294967548 rc 0/0 00000100:00100000:1.0:1788845016.277415:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 288 00002000:00100000:1.0:1788845016.277430:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c1f80 x1875739641364480/t4294967548(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:122/0 lens 488/448 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.277440:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c1f80 with x1875739641364480 00000100:00100000:1.0:1788845016.283495:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641364608 00000100:00100000:1.0:1788845016.283517:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641364608 00000100:00100000:1.0:1788845016.283591:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 578 00000100:00100000:1.0:1788845016.283602:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641364608:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845016.347513:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967548 is committed 00000100:00100000:1.0:1788845016.347651:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641364608:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 64045us (64396us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845016.347663:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 578 00000100:00100000:3.0:1788845016.353402:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641364736 00000100:00100000:3.0:1788845016.353412:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641364736 00000100:00100000:3.0:1788845016.353491:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 350 00000100:00100000:3.0:1788845016.353501:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641364736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845016.353522:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845016.353540:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba7d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7578 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.353604:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660163200:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845016.353710:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845016.356986:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660163200:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845016.357042:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba7d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7578 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.357066:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287aba7d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7578 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.357081:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845016.357150:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641364736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3649us (4107us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845016.357159:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 350 00000100:00100000:1.0:1788845016.370982:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641364864 00000100:00100000:1.0:1788845016.370993:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641364864 00000100:00100000:1.0:1788845016.371063:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 351 00000100:00100000:1.0:1788845016.371074:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641364864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.371092:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.371111:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f395d9d5/0xb2187301287aba84 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a757f expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.371187:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660163328:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845016.371275:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a012d330c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.381445:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660163328:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845016.381494:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f395d9d5/0xb2187301287aba84 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a757f expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.381520:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f395d9d5/0xb2187301287aba84 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a757f expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.381533:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3600, rc 0) 00000100:00100000:1.0:1788845016.381662:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641364864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10589us (10829us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845016.381671:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 351 00000100:00100000:3.0:1788845016.384574:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641364992 00000100:00100000:3.0:1788845016.384584:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641364992 00000100:00100000:3.0:1788845016.384650:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 579 00000100:00100000:3.0:1788845016.384661:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641364992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845016.384682:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845016.384701:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba8b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7586 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.384754:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.384795:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660163456:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845016.384872:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00000004:00010000:3.0:1788845016.390335:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845014 -> 1788845016 00000004:00010000:3.0:1788845016.390344:0:11036:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 584 -> 712 00000100:00100000:3.0:1788845016.390804:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660163456:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845016.390834:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845016.390855:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba8b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7586 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.390877:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845016.390881:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845016.391112:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aba8b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7586 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.391138:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845016.391354:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641364992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6693us (6899us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845016.391438:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 579 00000100:00100000:0.0:1788845016.395566:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641365120 00000100:00100000:0.0:1788845016.395580:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641365120 00000100:00100000:0.0:1788845016.395702:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 580 00000100:00100000:0.0:1788845016.395714:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641365120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.395736:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.395759:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a758d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.395795:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.395806:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.395841:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.395860:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 10 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.395923:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660163584:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845016.396028:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845016.401158:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660163584:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845016.401197:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.401217:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845016.401220:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845016.401224:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.401235:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b800, rc 0) 00000100:00100000:0.0:1788845016.401306:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641365120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5594us (7203us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845016.401315:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 580 00000100:00100000:1.0:1788845016.403810:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641365248 00000100:00100000:1.0:1788845016.403820:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641365248 00010000:00010000:1.0:1788845016.403861:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.403987:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 209 00000100:00100000:1.0:1788845016.403999:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641365248:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845016.404016:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845016.404042:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.404060:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.404083:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287aba76 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7571 expref: 11 pid: 5839 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.404098:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845016.404104:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.404116:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.404151:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.404713:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.404729:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660163712:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845016.404825:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a012d330c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845016.410493:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641365376 00000100:00100000:3.0:1788845016.410504:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641365376 00002000:00100000:3.0:1788845016.410553:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9f80 x1875739641365376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.410596:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9f80 with x1875739641365376 00000100:00100000:3.0:1788845016.410625:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 289 00000100:00100000:3.0:1788845016.410636:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641365376:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.411349:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641365376:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 711us (1403us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.411359:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 289 00002000:00100000:3.0:1788845016.411368:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9f80 x1875739641365376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.411376:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9f80 with x1875739641365376 00000100:00100000:1.0:1788845016.411785:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660163712:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845016.411812:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845016.411847:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641365248:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7850us (8112us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845016.411856:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 209 00000100:00100000:3.0:1788845016.416449:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641365504 00000100:00100000:3.0:1788845016.416459:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641365504 00002000:00100000:3.0:1788845016.416512:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fb800 x1875739641365504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.416523:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fb800 with x1875739641365504 00000100:00100000:3.0:1788845016.416564:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 290 00000100:00100000:3.0:1788845016.416574:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641365504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.417122:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641365504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 436us (839us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.417141:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 290 00002000:00100000:3.0:1788845016.417150:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fb800 x1875739641365504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.417167:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fb800 with x1875739641365504 00000100:00100000:0.0:1788845016.423540:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.423745:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641365632 00000100:00100000:3.0:1788845016.423769:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641365632 00002000:00100000:3.0:1788845016.423819:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c3800 x1875739641365632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.423831:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c3800 with x1875739641365632 00000100:00100000:3.0:1788845016.423872:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 291 00000100:00100000:3.0:1788845016.423912:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641365632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.424377:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641365632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 494us (844us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.424387:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 291 00002000:00100000:3.0:1788845016.424395:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c3800 x1875739641365632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.424403:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c3800 with x1875739641365632 00000100:00100000:1.0:1788845016.430719:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641365760 00000100:00100000:1.0:1788845016.430729:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641365760 00002000:00100000:1.0:1788845016.430779:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa300 x1875739641365760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.430792:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa300 with x1875739641365760 00000100:00100000:1.0:1788845016.430830:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 292 00000100:00100000:1.0:1788845016.430841:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641365760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845016.431093:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641365760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (513us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845016.431103:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 292 00002000:00100000:1.0:1788845016.431111:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa300 x1875739641365760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.431144:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa300 with x1875739641365760 00000100:00100000:0.0:1788845016.439499:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641366016 00000100:00100000:0.0:1788845016.439510:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641366016 00000100:00100000:0.0:1788845016.439571:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 352 00000100:00100000:0.0:1788845016.439592:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641366016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.439611:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.439629:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287aba99 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a759b expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.439690:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660163840:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845016.439784:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.439859:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641365888 00000100:00100000:1.0:1788845016.439863:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641365888 00000100:00100000:1.0:1788845016.439940:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 581 00000100:00100000:1.0:1788845016.439958:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641365888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.439968:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.439981:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abaa0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7594 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.440009:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abaa0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7594 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845016.440030:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845016.440038:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abaa0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7594 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.440052:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845016.440055:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845016.440059:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3800, rc 0) 00000100:00100000:1.0:1788845016.440088:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641365888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 132us (861us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845016.440095:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 581 00002000:00010000:3.0:1788845016.442951:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845015 -> 1788845016 00000100:00100000:3.0:1788845016.442981:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660163840:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845016.443027:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287aba99 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a759b expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.443051:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287aba99 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a759b expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.443065:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b400, rc 0) 00000100:00100000:3.0:1788845016.443125:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641366016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3545us (3841us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845016.443143:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 352 00000100:00100000:0.0:1788845016.453869:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641366144 00000100:00100000:0.0:1788845016.453879:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641366144 00000100:00100000:0.0:1788845016.453962:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 353 00000100:00100000:0.0:1788845016.453971:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641366144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.453988:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.454004:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75a2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.454041:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.454051:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 6) 00010000:00010000:0.0:1788845016.454055:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 6) 00010000:00010000:0.0:1788845016.454061:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1253376->1257471] (req 1253376->1257471) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a2 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.454091:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.454108:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 5 pid: 6682 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.454122:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660163968:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845016.454210:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845016.462458:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641366272 00000100:00100000:3.0:1788845016.462468:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641366272 00010000:00010000:3.0:1788845016.462514:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.462564:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 210 00000100:00100000:3.0:1788845016.462574:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641366272:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845016.462599:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845016.462623:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.462643:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.462670:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 6) 00010000:00010000:3.0:1788845016.462675:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 6) 00010000:00010000:3.0:1788845016.462684:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1253376->1257471) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a2 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.462695:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1253376->1257471) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a2 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.462739:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1253376->1257471) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a2 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.462755:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660164096:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845016.462860:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00010000:00010000:0.0:1788845016.464957:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287aba45 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7540 expref: 6 pid: 6682 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.464997:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660163968:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845016.465040:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1253376->1257471) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a2 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.465057:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a200, rc 0) 00000100:00100000:0.0:1788845016.465123:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641366144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11152us (11904us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845016.465141:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 353 00000100:00100000:1.0:1788845016.472733:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641366400 00000100:00100000:1.0:1788845016.472743:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641366400 00002000:00100000:1.0:1788845016.472797:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1691647) req@ffff94a0346f9880 x1875739641366400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.472829:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0346f9880 with x1875739641366400 ext(1048576->1691647) 00002000:00010000:1.0:1788845016.472835:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0346f9880 00000100:00100000:1.0:1788845016.472869:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 169 00000100:00100000:1.0:1788845016.472879:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641366400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.475348:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660164096:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845016.475377:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845016.475421:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641366272:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 12848us (13205us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845016.475432:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 210 00000100:00100000:1.0:1788845016.490518:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641366400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17628us (18020us total) trans 0 rc 643072/643072 00000100:00100000:1.0:1788845016.490532:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 169 00002000:00100000:1.0:1788845016.490545:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1691647) req@ffff94a0346f9880 x1875739641366400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:122/0 lens 488/440 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 643072/643072 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.490556:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0346f9880 with x1875739641366400 ext(1048576->1691647) 00002000:00010000:1.0:1788845016.490562:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0346f9880 00000100:00100000:0.0:1788845016.508606:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.508730:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641366528 00000100:00100000:3.0:1788845016.508741:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641366528 00002000:00100000:3.0:1788845016.508795:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1294335) req@ffff94a03f6c2a00 x1875739641366528/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.508808:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c2a00 with x1875739641366528 ext(1253376->1294335) 00002000:00010000:3.0:1788845016.508814:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c2a00 00000100:00100000:3.0:1788845016.508853:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 170 00000100:00100000:3.0:1788845016.508864:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641366528:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845016.511961:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641366528:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3078us (3344us total) trans 4294967360 rc 0/0 00000100:00100000:3.0:1788845016.511975:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 170 00002000:00100000:3.0:1788845016.511988:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1294335) req@ffff94a03f6c2a00 x1875739641366528/t4294967360(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:122/0 lens 488/448 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.512114:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c2a00 with x1875739641366528 ext(1253376->1294335) 00002000:00010000:3.0:1788845016.512121:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c2a00 00000100:00100000:0.0:1788845016.524169:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.524269:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641366656 00000100:00100000:3.0:1788845016.524280:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641366656 00000100:00100000:3.0:1788845016.524370:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 354 00000100:00100000:3.0:1788845016.524379:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641366656:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:1.0:1788845016.607288:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641366784 00000100:00100000:1.0:1788845016.607298:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641366784 00000100:00100000:1.0:1788845016.607361:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 355 00000100:00100000:1.0:1788845016.607371:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641366784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.607391:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.607408:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f395d9d5/0xb2187301287abaae lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75a9 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.607465:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035ef9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660164224:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845016.607572:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a012d330c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845016.610124:0:6685:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 341 -> 423 00000100:00100000:1.0:1788845016.610162:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035ef9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660164224:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845016.610218:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f395d9d5/0xb2187301287abaae lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.610242:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f395d9d5/0xb2187301287abaae lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.610256:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3600, rc 0) 00000100:00100000:1.0:1788845016.610315:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641366784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2945us (3142us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845016.610324:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 355 00000100:00100000:3.0:1788845016.617536:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641366912 00000100:00100000:3.0:1788845016.617548:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641366912 00002000:00100000:3.0:1788845016.617616:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359ad500 x1875739641366912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.617627:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359ad500 with x1875739641366912 00000100:00100000:3.0:1788845016.617668:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 293 00000100:00100000:3.0:1788845016.617678:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641366912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.617964:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641366912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 285us (522us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.617972:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 293 00002000:00100000:3.0:1788845016.617979:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359ad500 x1875739641366912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.617987:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359ad500 with x1875739641366912 00000100:00100000:1.0:1788845016.629616:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641367040 00000100:00100000:1.0:1788845016.629626:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641367040 00002000:00100000:1.0:1788845016.629679:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91500 x1875739641367040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.629691:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91500 with x1875739641367040 00000100:00100000:1.0:1788845016.629727:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 294 00000100:00100000:1.0:1788845016.629737:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641367040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845016.630036:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641367040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 300us (552us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845016.630044:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 294 00002000:00100000:1.0:1788845016.630052:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91500 x1875739641367040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.630060:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91500 with x1875739641367040 00000100:00100000:3.0:1788845016.643847:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641367168 00000100:00100000:3.0:1788845016.643857:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641367168 00002000:00100000:3.0:1788845016.643923:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91180 x1875739641367168/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.643934:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91180 with x1875739641367168 00000100:00100000:3.0:1788845016.643974:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 295 00000100:00100000:3.0:1788845016.643984:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641367168:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.644239:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641367168:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (498us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.644248:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 295 00002000:00100000:3.0:1788845016.644255:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91180 x1875739641367168/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.644270:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91180 with x1875739641367168 00000100:00100000:1.0:1788845016.653556:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641367296 00000100:00100000:1.0:1788845016.653582:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641367296 00002000:00100000:1.0:1788845016.653647:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93480 x1875739641367296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.653659:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93480 with x1875739641367296 00000100:00100000:1.0:1788845016.653691:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 296 00000100:00100000:1.0:1788845016.653702:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641367296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845016.654231:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641367296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 529us (899us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845016.654240:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 296 00002000:00100000:1.0:1788845016.654248:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93480 x1875739641367296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.654257:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93480 with x1875739641367296 00010000:00010000:0.0:1788845016.658623:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845016.668662:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641367424 00000100:00100000:3.0:1788845016.668673:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641367424 00002000:00100000:3.0:1788845016.668727:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d92680 x1875739641367424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.668739:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d92680 with x1875739641367424 00000100:00100000:3.0:1788845016.668927:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 297 00000100:00100000:3.0:1788845016.668937:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641367424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.673121:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641367552 00000100:00100000:3.0:1788845016.673142:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641367552 00002000:00100000:3.0:1788845016.673208:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93b80 x1875739641367552/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.673219:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93b80 with x1875739641367552 00000100:00100000:3.0:1788845016.673323:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 298 00000100:00100000:3.0:1788845016.673336:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641367552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.673583:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641367552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 249us (566us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.673591:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 298 00002000:00100000:3.0:1788845016.673599:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93b80 x1875739641367552/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.673606:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93b80 with x1875739641367552 00000100:00100000:0.0:1788845016.683524:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.683791:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641367680 00000100:00100000:3.0:1788845016.683800:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641367680 00002000:00100000:3.0:1788845016.683850:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ed80 x1875739641367680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.683874:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ed80 with x1875739641367680 00000100:00100000:3.0:1788845016.683924:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 299 00000100:00100000:3.0:1788845016.683934:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641367680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.684355:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641367680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 421us (841us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.684363:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 299 00002000:00100000:3.0:1788845016.684370:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ed80 x1875739641367680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.684378:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ed80 with x1875739641367680 00000100:00100000:1.0:1788845016.692018:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641367808 00000100:00100000:1.0:1788845016.692028:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641367808 00002000:00100000:1.0:1788845016.692079:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90700 x1875739641367808/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.692090:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90700 with x1875739641367808 00000100:00100000:1.0:1788845016.692135:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 300 00000100:00100000:1.0:1788845016.692146:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641367808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845016.692440:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641367808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 295us (571us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845016.692448:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 300 00002000:00100000:1.0:1788845016.692457:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90700 x1875739641367808/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.692476:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90700 with x1875739641367808 00000100:00100000:0.0:1788845016.700521:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.712557:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.712714:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641368064 00000100:00100000:1.0:1788845016.712724:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641368064 00002000:00100000:1.0:1788845016.712777:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b7480 x1875739641368064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.712787:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7480 with x1875739641368064 00000100:00100000:1.0:1788845016.712824:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 302 00000100:00100000:1.0:1788845016.712834:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641368064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845016.713150:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641368064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 316us (601us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845016.713158:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 302 00002000:00100000:1.0:1788845016.713166:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b7480 x1875739641368064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.713172:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7480 with x1875739641368064 00000100:00100000:1.0:1788845016.718292:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.724035:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.725785:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641368320 00000100:00100000:1.0:1788845016.725800:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641368320 00002000:00100000:1.0:1788845016.725854:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48e00 x1875739641368320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.725865:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48e00 with x1875739641368320 00000100:00100000:1.0:1788845016.725936:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 304 00000100:00100000:1.0:1788845016.725945:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641368320:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845016.726450:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641368320:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 503us (2424us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845016.726473:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 304 00002000:00100000:1.0:1788845016.726481:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48e00 x1875739641368320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.726488:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48e00 with x1875739641368320 00000100:00100000:0.0:1788845016.734509:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.734662:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641368448 00000100:00100000:3.0:1788845016.734672:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641368448 00002000:00100000:3.0:1788845016.734720:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48a80 x1875739641368448/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.734731:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48a80 with x1875739641368448 00000100:00100000:3.0:1788845016.734770:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 305 00000100:00100000:3.0:1788845016.734780:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641368448:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.735081:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641368448:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 300us (580us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.735089:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 305 00002000:00100000:3.0:1788845016.735096:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48a80 x1875739641368448/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.735105:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48a80 with x1875739641368448 00000100:00100000:3.0:1788845016.745420:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641368576 00000100:00100000:3.0:1788845016.745447:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641368576 00002000:00100000:3.0:1788845016.745503:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d92d80 x1875739641368576/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.745516:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d92d80 with x1875739641368576 00000100:00100000:3.0:1788845016.745558:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 306 00000100:00100000:3.0:1788845016.745568:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641368576:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.745854:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641368576:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 286us (790us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.745864:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 306 00002000:00100000:3.0:1788845016.745871:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d92d80 x1875739641368576/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.745879:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d92d80 with x1875739641368576 00000100:00100000:0.0:1788845016.753222:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.753371:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641368704 00000100:00100000:3.0:1788845016.753381:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641368704 00002000:00100000:3.0:1788845016.753436:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48700 x1875739641368704/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.753446:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48700 with x1875739641368704 00000100:00100000:3.0:1788845016.753530:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 307 00000100:00100000:3.0:1788845016.753540:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641368704:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.753994:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641368704:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 454us (780us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.754003:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 307 00002000:00100000:3.0:1788845016.754010:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48700 x1875739641368704/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.754036:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48700 with x1875739641368704 00000100:00100000:0.0:1788845016.757387:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.757480:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641368832 00000100:00100000:3.0:1788845016.757489:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641368832 00002000:00100000:3.0:1788845016.757533:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48000 x1875739641368832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.757541:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48000 with x1875739641368832 00000100:00100000:3.0:1788845016.757572:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 308 00000100:00100000:3.0:1788845016.757582:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641368832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.758204:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641368832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 620us (823us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845016.758212:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 308 00002000:00100000:3.0:1788845016.758221:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48000 x1875739641368832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/4536 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845016.758230:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48000 with x1875739641368832 00000100:00100000:1.0:1788845016.763278:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.768508:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.768962:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641369088 00000100:00100000:1.0:1788845016.768969:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641369088 00010000:00010000:1.0:1788845016.769015:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 15 pid: 5841 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.769046:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 211 00000100:00100000:1.0:1788845016.769055:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641369088:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845016.769070:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845016.769098:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 15 pid: 5841 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.769114:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 15 pid: 5841 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.769147:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287aba92 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a758d expref: 15 pid: 5841 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.769163:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845016.769168:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abab5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b0 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.769207:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abab5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b0 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.769216:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abab5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b0 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.769251:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abab5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b0 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.769276:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660164480:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845016.769332:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a012d330c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.772022:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660164480:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845016.772054:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845016.772107:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641369088:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3052us (3613us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845016.772118:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 211 00000100:00100000:3.0:1788845016.774919:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.780071:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.780630:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641369472 00000100:00100000:1.0:1788845016.780640:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641369472 00000100:00100000:1.0:1788845016.780811:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 583 00000100:00100000:1.0:1788845016.780823:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641369472:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845016.780876:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845016.780945:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023af000/0xb2187301287ab9ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74ad expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.780966:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845016.780985:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845016.781348:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287ababc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.781363:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aee00/0xb2187301287ab9b9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a74b4 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.781373:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287ababc lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.781406:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aee00/0xb2187301287ab9b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74b4 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.781424:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aee00/0xb2187301287ab9b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74b4 expref: 11 pid: 5840 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.781440:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660164608:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845016.781539:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845016.785581:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.786393:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788845016.786412:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035928380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660164736:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845016.786473:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1788845016.788561:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f023aee00/0xb2187301287ab9b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a74b4 expref: 11 pid: 5840 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.788583:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660164608:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845016.788610:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287ababc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.788620:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287ababc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.788626:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287ababc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845016.788820:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967550 00000004:00010000:1.0:1788845016.788866:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788845016.788869:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845016 -> 1788845016 00000004:00010000:1.0:1788845016.788872:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845016 -> 1788845016 00000004:00010000:1.0:1788845016.788875:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845016 -> 1788845016 00000004:00010000:1.0:1788845016.788877:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:1.0:1788845016.788991:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c4285808 reply state 00000000530113d9 transno 4294967551 ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2400/0xb2187301287ababc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.789014:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845016.789115:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641369472:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8293us (9055us total) trans 4294967551 rc 0/0 00000100:00100000:1.0:1788845016.789131:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 583 00000020:00080000:1.0:1788845016.789137:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845016.792840:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.797164:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.797576:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641369856 00000100:00100000:1.0:1788845016.797583:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641369856 00010000:00010000:1.0:1788845016.797615:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abab5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b0 expref: 11 pid: 5840 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.797658:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 213 00000100:00100000:1.0:1788845016.797667:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641369856:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845016.797680:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845016.797702:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abab5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b0 expref: 11 pid: 5840 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.797717:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abab5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b0 expref: 11 pid: 5840 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.797744:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abab5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b0 expref: 11 pid: 5840 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.797758:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845016.797762:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.797770:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.797777:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.797808:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.797822:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660164992:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845016.797872:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.801161:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660164992:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845016.801194:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845016.801253:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641369856:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3586us (4097us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845016.801264:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 213 00000100:00100000:0.0:1788845016.806913:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.807034:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641369984 00000100:00100000:3.0:1788845016.807045:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641369984 00000100:00100000:3.0:1788845016.807105:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 356 00000100:00100000:3.0:1788845016.807116:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641369984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845016.807145:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845016.807166:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abaca lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75be expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.807214:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1253376->1257471) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a2 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.807223:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(5 6) 00010000:00010000:3.0:1788845016.807227:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 6) 00010000:00010000:3.0:1788845016.807233:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abaca lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75be expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.807269:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1253376->1257471) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a2 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.807286:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abaa7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1253376->1257471) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75a2 expref: 5 pid: 6682 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.807300:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660165120:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845016.807386:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845016.817861:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.818135:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660165120:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845016.818172:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abaca lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75be expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.818196:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845016.818264:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641369984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11148us (11393us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845016.818281:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 356 00000100:00100000:3.0:1788845016.832620:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.832796:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641370240 00000100:00100000:1.0:1788845016.832804:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641370240 00002000:00010000:1.0:1788845016.832853:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1356146->18446744073709551615) 00002000:00100000:1.0:1788845016.832858:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035929c00 with x1875739641370240 ext(1356146->18446744073709551615) 00002000:00010000:1.0:1788845016.832870:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035929c00. 00000100:00100000:1.0:1788845016.832934:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 171 00000100:00100000:1.0:1788845016.832943:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641370240:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845016.833521:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1691448 -> 1356146 00002000:00010000:1.0:1788845016.833526:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845016 -> 1788845016 00002000:00010000:1.0:1788845016.833529:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845016 -> 1788845016 00002000:00010000:1.0:1788845016.833531:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845016 -> 1788845016 00002000:00010000:1.0:1788845016.833533:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 423 -> 423 00000100:00100000:1.0:1788845016.833578:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641370240:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 634us (957us total) trans 4294967361 rc 0/0 00000100:00100000:1.0:1788845016.833586:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 171 00002000:00010000:1.0:1788845016.833590:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1356146->18446744073709551615) 00002000:00100000:1.0:1788845016.833593:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035929c00 with x1875739641370240 ext(1356146->18446744073709551615) 00002000:00010000:1.0:1788845016.833598:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035929c00. 00000100:00100000:3.0:1788845016.841238:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.841392:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641370368 00000100:00100000:1.0:1788845016.841404:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641370368 00000100:00100000:1.0:1788845016.841474:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 585 00000100:00100000:1.0:1788845016.841484:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641370368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.841507:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.841544:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2000/0xb2187301287abad1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75c5 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.841657:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.841690:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.841704:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845016.841731:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1356146 00000004:00200000:1.0:1788845016.841737:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845016.841755:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.841781:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2000/0xb2187301287abad1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.841798:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.841808:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845016.841811:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845016.841814:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3400, rc 0) 00000100:00100000:1.0:1788845016.841878:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641370368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 395us (649us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845016.841949:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 585 00000100:00100000:3.0:1788845016.852553:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.852678:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641370496 00000100:00100000:1.0:1788845016.852689:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641370496 00002000:00100000:1.0:1788845016.852737:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1359871) req@ffff94a03592a680 x1875739641370496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.852749:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03592a680 with x1875739641370496 ext(1048576->1359871) 00002000:00010000:1.0:1788845016.852782:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03592a680 00000100:00100000:1.0:1788845016.852815:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 172 00000100:00100000:1.0:1788845016.852825:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641370496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845016.857730:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.857854:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641370624 00000100:00100000:3.0:1788845016.857864:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641370624 00000100:00100000:3.0:1788845016.858007:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 275 00000100:00100000:3.0:1788845016.858018:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641370624:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845016.858095:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641370624:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 79us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845016.858103:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 275 00000100:00100000:3.0:1788845016.860397:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.863266:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641370752 00000100:00100000:1.0:1788845016.863281:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641370752 00002000:00100000:1.0:1788845016.863334:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03592b800 x1875739641370752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/0 e 0 to 0 dl 1788845027 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.863346:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03592b800 with x1875739641370752 00000100:00100000:1.0:1788845016.863383:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 310 00000100:00100000:1.0:1788845016.863392:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641370752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845016.871456:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.871733:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641370880 00000100:00100000:1.0:1788845016.871743:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641370880 00000100:00100000:1.0:1788845016.871814:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 586 00000100:00100000:1.0:1788845016.871825:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641370880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.871850:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.871872:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287abadf lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75cc expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.872222:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3c00/0xb2187301287abae6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.872240:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3c00/0xb2187301287abae6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.872252:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3c00/0xb2187301287abae6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845016.872281:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1356146 00000004:00200000:1.0:1788845016.872298:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845016.872321:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3c00/0xb2187301287abae6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.872347:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2200/0xb2187301287abadf lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75cc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.872365:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3c00/0xb2187301287abae6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75cc expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.872374:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845016.872377:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845016.872381:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3c00, rc 0) 00000100:00100000:1.0:1788845016.872451:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641370880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 627us (1006us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845016.872458:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 586 00000100:00100000:3.0:1788845016.878815:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.878866:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.879180:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641371008 00000100:00100000:1.0:1788845016.879196:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641371008 00000100:00100000:1.0:1788845016.879275:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 587 00000100:00100000:1.0:1788845016.879286:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641371008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.879318:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.879341:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e2600/0xb2187301287abaed lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75d3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.879379:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.879421:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660165376:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845016.879522:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845016.884500:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641371136 00000100:00100000:0.0:1788845016.884515:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641371136 00000100:00100000:0.0:1788845016.884600:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 357 00000100:00100000:0.0:1788845016.884612:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641371136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.884637:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.884662:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287abaf4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75da expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.884741:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f22680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660165504:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845016.884979:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.895581:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641370752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 32178us (35181us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1788845016.895604:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 310 00002000:00100000:1.0:1788845016.895634:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03592b800 x1875739641370752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:122/0 lens 488/440 e 0 to 0 dl 1788845027 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845016.895647:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03592b800 with x1875739641370752 00000100:00100000:0.0:1788845016.906328:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f22680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660165504:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845016.906386:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287abaf4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75da expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.906419:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e1c942f7/0xb2187301287abaf4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75da expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.906437:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b200, rc 0) 00000100:00100000:0.0:1788845016.906601:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641371136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 21989us (27732us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845016.906611:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 357 00000100:00100000:0.0:1788845016.912319:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.912370:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.912518:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641371392 00000100:00100000:3.0:1788845016.912532:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641371392 00000100:00100000:3.0:1788845016.912607:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 358 00000100:00100000:3.0:1788845016.912620:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641371392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845016.912647:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845016.912673:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abafb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75e8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.912754:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660165632:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845016.912867:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845016.915939:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660165632:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845016.916011:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abafb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75e8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.916052:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abafb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75e8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.916074:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:3.0:1788845016.916160:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641371392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3542us (3788us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845016.916172:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 358 00000100:00100000:1.0:1788845016.924680:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.924782:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.924965:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641371648 00000100:00100000:0.0:1788845016.924998:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641371648 00000100:00100000:0.0:1788845016.925082:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 359 00000100:00100000:0.0:1788845016.925095:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641371648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.925129:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.925155:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000843b68c1/0xb2187301287abb09 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75f6 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.925234:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660165888:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845016.925360:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845016.925853:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641371520 00000100:00100000:3.0:1788845016.925860:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641371520 00000100:00100000:0.0:1788845016.927209:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 589 00000100:00100000:0.0:1788845016.927226:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641371520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.927260:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.927288:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a800/0xb2187301287abb10 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75ef expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.927347:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.927399:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f22d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660166016:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845016.927501:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845016.929404:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660165888:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845016.929455:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000843b68c1/0xb2187301287abb09 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75f6 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.929485:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000843b68c1/0xb2187301287abb09 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75f6 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.929504:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a400, rc 0) 00000100:00100000:3.0:1788845016.929574:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641371648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4481us (4790us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845016.929584:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 359 00000100:00100000:0.0:1788845016.938523:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f22d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660166016:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845016.938564:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845016.938592:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a800/0xb2187301287abb10 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75ef expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.938620:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845016.938645:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845016.938652:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a800/0xb2187301287abb10 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75ef expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.938669:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a800, rc 0) 00000100:00100000:0.0:1788845016.938756:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641371520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11535us (14091us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845016.938766:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 589 00000100:00100000:0.0:1788845016.950471:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.950623:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845016.950732:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641371904 00000100:00100000:1.0:1788845016.950745:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641371904 00000100:00100000:1.0:1788845016.950825:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 360 00000100:00100000:1.0:1788845016.950838:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641371904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845016.950863:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845016.950933:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287abb17 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7604 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845016.951025:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ced880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660166144:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845016.951147:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845016.952378:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641371776 00000100:00100000:3.0:1788845016.952409:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641371776 00010000:00010000:0.0:1788845016.952851:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251bc00/0xb2187301287abb1e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a75fd expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.953104:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.953165:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660166272:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845016.953382:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00000100:00100000:1.0:1788845016.959399:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ced880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660166144:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845016.959444:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287abb17 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7604 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.959464:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000093565e1/0xb2187301287abb17 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7604 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845016.959489:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2600, rc 0) 00000100:00100000:1.0:1788845016.959514:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641371904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8680us (8892us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845016.959521:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 360 00000100:00100000:0.0:1788845016.966154:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.966375:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.966503:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641372032 00000100:00100000:0.0:1788845016.966508:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641372032 00000100:00100000:0.0:1788845016.966542:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 591 00000100:00100000:0.0:1788845016.966550:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641372032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845016.966561:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845016.966576:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b600/0xb2187301287abb2c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a760b expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845016.966638:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845016.966667:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660166528:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845016.966696:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845016.982360:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.982416:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.983664:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641372416 00000100:00100000:3.0:1788845016.983673:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641372416 00000100:00100000:3.0:1788845016.983730:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 362 00000100:00100000:3.0:1788845016.983740:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641372416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845016.983757:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845016.983772:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb3a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7620 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.983824:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660166784:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845016.984790:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845016.989352:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660166784:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845016.989412:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb3a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7620 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.989447:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb3a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7620 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845016.989464:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:3.0:1788845016.989548:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641372416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5809us (7131us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845016.989557:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 362 00000100:00100000:0.0:1788845016.996338:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845016.996400:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845016.998859:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641372672 00000100:00100000:3.0:1788845016.998877:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641372672 00000100:00100000:3.0:1788845016.998971:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 363 00000100:00100000:3.0:1788845016.998984:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641372672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845016.999206:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845016.999229:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb48 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a762e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845016.999462:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660167040:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845016.999619:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845017.005162:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660167040:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845017.005190:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb48 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a762e expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.005210:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb48 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a762e expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.005220:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:3.0:1788845017.005266:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641372672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6286us (8866us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845017.005275:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 363 00000100:00100000:0.0:1788845017.010793:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00080000:3.0:1788845017.011088:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845017.020374:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.021934:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641372928 00000100:00100000:0.0:1788845017.021947:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641372928 00000100:00100000:0.0:1788845017.022038:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 364 00000100:00100000:0.0:1788845017.022050:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641372928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845017.022072:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845017.022095:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000acab16e4/0xb2187301287abb56 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a763c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845017.022182:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660167296:192.168.201.29@tcp:106: 00000100:00080000:3.0:1788845017.022535:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845017.022680:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845017.027744:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660167296:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845017.027803:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000acab16e4/0xb2187301287abb56 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a763c expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.027862:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000acab16e4/0xb2187301287abb56 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a763c expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.027879:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a600, rc 0) 00000100:00100000:0.0:1788845017.028018:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641372928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5970us (7656us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845017.028028:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 364 00000100:00100000:1.0:1788845017.032351:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.032869:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641373056 00000100:00100000:0.0:1788845017.032904:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641373056 00000100:00100000:0.0:1788845017.032985:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 595 00000100:00100000:0.0:1788845017.033009:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641373056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845017.033038:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845017.033064:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abb5d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7643 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.033117:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845017.033184:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660167424:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845017.033300:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845017.039947:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.043330:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660167424:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845017.043367:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845017.043393:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abb5d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7643 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.043445:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845017.043449:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845017.043455:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abb5d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7643 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.043473:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a600, rc 0) 00000100:00100000:0.0:1788845017.043551:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641373056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10555us (11217us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845017.043561:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 595 00000100:00100000:1.0:1788845017.051436:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.051532:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.051775:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641373312 00000100:00100000:0.0:1788845017.051789:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641373312 00000100:00100000:0.0:1788845017.051873:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 596 00000100:00100000:0.0:1788845017.051909:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641373312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845017.051938:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845017.051969:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abb6b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7651 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.052029:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845017.052081:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660167680:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845017.052194:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845017.068604:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.068670:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.069617:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641373696 00000100:00100000:3.0:1788845017.069624:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641373696 00000100:00100000:3.0:1788845017.069718:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 367 00000100:00100000:3.0:1788845017.069727:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641373696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.069741:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.069756:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb80 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7666 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.069824:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660168064:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.069860:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000004:00010000:0.0:1788845017.073549:0:5839:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845016 -> 1788845017 00000004:00010000:0.0:1788845017.073579:0:5839:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845016 -> 1788845017 00000100:00100000:0.0:1788845017.073615:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660167936:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845017.073666:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845017.073693:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abb79 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a765f expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.073719:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845017.073722:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845017.073727:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abb79 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a765f expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.073745:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:0.0:1788845017.073822:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641373568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4948us (5150us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845017.073832:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 597 00002000:00010000:3.0:1788845017.075790:0:6685:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845016 -> 1788845017 00002000:00010000:3.0:1788845017.075802:0:6685:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845016 -> 1788845017 00000100:00100000:3.0:1788845017.075836:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660168064:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845017.075880:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb80 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7666 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.075933:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abb80 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7666 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.075947:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:3.0:1788845017.076141:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641373696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6413us (7548us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845017.076152:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 367 00000100:00100000:1.0:1788845017.080514:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.080589:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.080772:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641373824 00000100:00100000:3.0:1788845017.080787:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641373824 00000100:00100000:3.0:1788845017.080866:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 598 00000100:00100000:3.0:1788845017.080878:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641373824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.080944:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.080972:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287abb87 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a766d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.081031:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.081103:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660168192:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.081209:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845017.081415:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641373952 00000100:00100000:0.0:1788845017.081421:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641373952 00000100:00100000:0.0:1788845017.081459:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 368 00000100:00100000:0.0:1788845017.081467:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641373952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845017.081478:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845017.081493:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009f62ed95/0xb2187301287abb8e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7674 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845017.081540:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660168320:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845017.081578:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845017.085201:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660168192:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845017.085440:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660168320:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845017.085479:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009f62ed95/0xb2187301287abb8e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7674 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.085512:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009f62ed95/0xb2187301287abb8e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7674 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.085528:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251be00, rc 0) 00002000:00010000:3.0:1788845017.085718:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845017.085735:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287abb87 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a766d expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.085751:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845017.085754:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845017.085759:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287abb87 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a766d expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.085772:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845017.085819:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641373952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4351us (5320us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845017.085830:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 368 00000100:00100000:3.0:1788845017.086714:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641373824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5763us (6049us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845017.086724:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 598 00000100:00100000:1.0:1788845017.092172:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.092288:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.094022:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641374080 00000100:00100000:0.0:1788845017.094036:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641374080 00000100:00100000:0.0:1788845017.094128:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 599 00000100:00100000:0.0:1788845017.094138:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641374080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845017.094161:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845017.094183:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abb9c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a767b expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.094230:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845017.094690:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660168576:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845017.094793:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845017.104010:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660168576:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845017.104057:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845017.104087:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abb9c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a767b expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.104117:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845017.104130:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845017.104137:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abb9c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a767b expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.104157:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251aa00, rc 0) 00000100:00100000:0.0:1788845017.104226:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641374080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10088us (11935us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845017.104235:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 599 00000100:00100000:1.0:1788845017.110217:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.115367:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.115970:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641374464 00000100:00100000:3.0:1788845017.115979:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641374464 00002000:00100000:3.0:1788845017.116073:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb6a00 x1875739641374464/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:123/0 lens 488/0 e 0 to 0 dl 1788845028 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845017.116086:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb6a00 with x1875739641374464 00010000:00010000:3.0:1788845017.116094:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 17 pid: 5841 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.116114:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 17 pid: 5841 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845017.116151:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a007eb6a00 00000100:00100000:3.0:1788845017.116187:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 15 00000100:00100000:3.0:1788845017.116198:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641374464:12345-192.168.201.29@tcp:4:fsx.0 00000100:00080000:0.0:1788845017.117810:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00010000:3.0:1788845017.122703:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845017.122714:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845017 -> 1788845017 00000004:00010000:3.0:1788845017.122718:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845016 -> 1788845016 00000004:00010000:3.0:1788845017.122720:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845017 -> 1788845017 00000004:00010000:3.0:1788845017.122723:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845017.122820:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641374464:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 6616us (7458us total) trans 4294967552 rc 0/0 00000100:00100000:3.0:1788845017.122833:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 15 00002000:00100000:3.0:1788845017.122849:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb6a00 x1875739641374464/t4294967552(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:123/0 lens 488/448 e 0 to 0 dl 1788845028 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845017.122860:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb6a00 with x1875739641374464 00010000:00010000:3.0:1788845017.122873:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 19 pid: 5841 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.123141:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abac3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75b7 expref: 19 pid: 5841 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845017.123152:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a007eb6a00 00000100:00100000:1.0:1788845017.130300:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.138856:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.162538:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.173434:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.180194:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.185026:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641375104 00000100:00100000:3.0:1788845017.185042:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641375104 00002000:00100000:0.0:1788845017.187385:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f21500 x1875739641375104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 488/0 e 0 to 0 dl 1788845028 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845017.187399:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f21500 with x1875739641375104 00000100:00100000:0.0:1788845017.187453:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 314 00000100:00100000:0.0:1788845017.187464:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641375104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845017.187742:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641375104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 279us (7563us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845017.187759:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 314 00002000:00100000:0.0:1788845017.187767:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f21500 x1875739641375104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 488/4536 e 0 to 0 dl 1788845028 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845017.187775:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f21500 with x1875739641375104 00000100:00100000:3.0:1788845017.197395:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.198268:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641375232 00000100:00100000:1.0:1788845017.198284:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641375232 00002000:00100000:1.0:1788845017.198364:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be5500 x1875739641375232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 488/0 e 0 to 0 dl 1788845028 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845017.198377:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be5500 with x1875739641375232 00000100:00100000:1.0:1788845017.198435:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 315 00000100:00100000:1.0:1788845017.198448:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641375232:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845017.199193:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641375232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 743us (2307us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845017.199204:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 315 00002000:00100000:1.0:1788845017.199212:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be5500 x1875739641375232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 488/4536 e 0 to 0 dl 1788845028 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845017.199221:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be5500 with x1875739641375232 00000100:00100000:1.0:1788845017.219497:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.219647:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641375360 00000100:00100000:3.0:1788845017.219660:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641375360 00002000:00100000:3.0:1788845017.219726:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb7100 x1875739641375360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 488/0 e 0 to 0 dl 1788845028 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845017.219740:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb7100 with x1875739641375360 00000100:00100000:3.0:1788845017.219790:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 316 00000100:00100000:3.0:1788845017.219802:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641375360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845017.240157:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.252705:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.260997:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.267060:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.276600:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.286091:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.303593:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.320382:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.320494:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641376384 00000100:00100000:3.0:1788845017.320508:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641376384 00000100:00100000:3.0:1788845017.320590:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 371 00000100:00100000:3.0:1788845017.320603:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641376384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.320627:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.320663:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287abbb1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7697 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.320740:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660169088:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.320850:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845017.323665:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660169088:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845017.323724:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287abbb1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7697 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.323757:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287abbb1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7697 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.323796:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:3.0:1788845017.323874:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641376384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3273us (3506us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845017.324135:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 371 00000100:00100000:0.0:1788845017.329777:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.339616:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.339772:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641376640 00000100:00100000:3.0:1788845017.339785:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641376640 00010000:00010000:3.0:1788845017.339854:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abaca lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75be expref: 7 pid: 6684 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.339981:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 216 00000100:00100000:3.0:1788845017.340004:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641376640:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845017.340026:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845017.340057:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abaca lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75be expref: 7 pid: 6684 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.340080:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abaca lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75be expref: 7 pid: 6684 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.340110:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abaca lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75be expref: 7 pid: 6684 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.340140:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 6) 00010000:00010000:3.0:1788845017.340146:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 6) 00010000:00010000:3.0:1788845017.340157:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287abbb8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1253376->1273855) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a769e expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.340187:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287abbb8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1253376->1273855) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a769e expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.340237:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287abbb8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1253376->1273855) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a769e expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.340265:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660169344:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845017.340371:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00080000:0.0:1788845017.341579:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845017.345020:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.345727:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660169344:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845017.345760:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845017.345823:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641376640:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5831us (6222us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845017.345836:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 216 00000100:00100000:1.0:1788845017.346376:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641376768 00000100:00100000:1.0:1788845017.346386:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641376768 00000100:00100000:1.0:1788845017.346462:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 173 00000100:00100000:1.0:1788845017.346474:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641376768:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:1.0:1788845017.347079:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1356146 -> 1356146 00002000:00010000:1.0:1788845017.347096:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845017 -> 1788845016 00002000:00010000:1.0:1788845017.347098:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845016 -> 1788845016 00002000:00010000:1.0:1788845017.347100:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845017 -> 1788845016 00002000:00010000:1.0:1788845017.347102:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 423 -> 423 00000100:00100000:1.0:1788845017.347165:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641376768:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 691us (2159us total) trans 4294967362 rc 0/0 00000100:00100000:1.0:1788845017.347173:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 173 00000100:00100000:0.0:1788845017.353667:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.356870:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.357282:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641377024 00000100:00100000:3.0:1788845017.357307:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641377024 00000100:00100000:3.0:1788845017.357372:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 373 00000100:00100000:3.0:1788845017.357385:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641377024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.357515:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.357535:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abbc6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a76ac expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.358311:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660169472:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.358532:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788845017.366201:0:6685:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845016 -> 1788845017 00002000:00010000:3.0:1788845017.366211:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845016 -> 1788845017 00002000:00010000:3.0:1788845017.366213:0:6685:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845016 -> 1788845017 00000100:00100000:3.0:1788845017.366239:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660169472:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845017.366279:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abbc6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ac expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.366487:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abbc6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ac expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.366503:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845017.366579:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641377024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9198us (9724us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845017.366588:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 373 00000100:00100000:0.0:1788845017.375048:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.375274:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641377152 00000100:00100000:3.0:1788845017.375284:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641377152 00000100:00100000:3.0:1788845017.375354:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 374 00000100:00100000:3.0:1788845017.375365:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641377152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.375382:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.375411:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abbcd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a76b3 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.375476:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660169600:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.375627:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845017.386432:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660169600:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845017.386516:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abbcd lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76b3 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.386548:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abbcd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76b3 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.386566:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845017.386660:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641377152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11295us (11620us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845017.386671:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 374 00000100:00100000:1.0:1788845017.403227:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.403272:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.403374:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641377280 00000100:00100000:3.0:1788845017.403392:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641377280 00002000:00100000:3.0:1788845017.403465:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb5c00 x1875739641377280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 504/0 e 0 to 0 dl 1788845028 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845017.403478:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb5c00 with x1875739641377280 00000100:00100000:3.0:1788845017.403535:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 323 00000100:00100000:3.0:1788845017.403550:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641377280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845017.441915:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641377280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 38324us (38653us total) trans 0 rc 999424/999424 00000100:00100000:0.0:1788845017.441951:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 323 00002000:00100000:0.0:1788845017.441965:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb5c00 x1875739641377280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 504/440 e 0 to 0 dl 1788845028 ref 1 fl Complete:/600/0 rc 999424/999424 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845017.441976:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb5c00 with x1875739641377280 00000100:00100000:3.0:1788845017.458229:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.458639:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641377664 00000100:00100000:1.0:1788845017.458653:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641377664 00000100:00100000:1.0:1788845017.458737:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 602 00000100:00100000:1.0:1788845017.458748:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641377664:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845017.458802:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845017.458830:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3c00/0xb2187301287abae6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75cc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.458850:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845017.458855:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845017.459128:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.459142:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.459151:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.459202:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.459220:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 18 pid: 11036 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845017.459234:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660169728:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845017.459325:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845017.469056:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.470487:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641377792 00000100:00100000:1.0:1788845017.470506:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641377792 00010000:00010000:1.0:1788845017.470580:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 19 pid: 11036 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845017.470634:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 217 00000100:00100000:1.0:1788845017.470643:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641377792:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845017.470659:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845017.470666:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 19 pid: 11036 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.470686:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 19 pid: 11036 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.470713:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3400/0xb2187301287abad8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a75c5 expref: 19 pid: 11036 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.470732:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845017.470755:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.470766:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.470773:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.470810:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845017.470874:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641377792:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 232us (1833us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845017.470914:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 217 00010000:00010000:3.0:1788845017.471376:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.471393:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845017.471575:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967553 00000004:00010000:3.0:1788845017.471664:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845017.471669:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845017 -> 1788845017 00000004:00010000:3.0:1788845017.471672:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845016 -> 1788845016 00000004:00010000:3.0:1788845017.471674:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845017 -> 1788845017 00000004:00010000:3.0:1788845017.471677:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:3.0:1788845017.471686:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c9d7cb90 reply state 0000000096bf5e7a transno 4294967554 ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.471697:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845017.471853:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641377664:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13103us (13636us total) trans 4294967554 rc 0/0 00000100:00100000:3.0:1788845017.471863:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 602 00000020:00080000:3.0:1788845017.471869:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845017.480946:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.480965:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.480973:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.480979:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f027e3800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845017.480984:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.480993:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.481013:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845017.481016:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845017.481020:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.481027:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3800/0xb2187301287abbd4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.482090:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.492352:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.494175:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641378048 00000100:00100000:3.0:1788845017.494194:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641378048 00010000:00010000:3.0:1788845017.494325:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abbbf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76a5 expref: 18 pid: 5839 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.494385:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 218 00000100:00100000:3.0:1788845017.494399:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641378048:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845017.494418:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845017.494457:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abbbf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76a5 expref: 18 pid: 5839 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.494479:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abbbf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76a5 expref: 18 pid: 5839 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.494530:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abbbf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76a5 expref: 18 pid: 5839 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.494551:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845017.494558:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.494570:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.494580:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.494630:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.494648:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660169984:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845017.494761:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845017.502048:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.502521:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660169984:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845017.502562:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845017.502662:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641378048:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8263us (10317us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845017.502676:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 218 00000100:00100000:1.0:1788845017.525342:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641378304 00000100:00100000:1.0:1788845017.525363:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641378304 00002000:00100000:1.0:1788845017.525447:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1359872->2047999) req@ffff94a019d90a80 x1875739641378304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 488/0 e 0 to 0 dl 1788845028 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845017.525460:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d90a80 with x1875739641378304 ext(1359872->2047999) 00002000:00010000:1.0:1788845017.525467:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d90a80 00000100:00100000:1.0:1788845017.525581:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 176 00000100:00100000:1.0:1788845017.525592:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641378304:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845017.553435:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641378304:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 27827us (31579us total) trans 0 rc 688128/688128 00000100:00100000:1.0:1788845017.553453:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 176 00002000:00100000:1.0:1788845017.553469:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1359872->2047999) req@ffff94a019d90a80 x1875739641378304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 488/440 e 0 to 0 dl 1788845028 ref 1 fl Complete:/600/0 rc 688128/688128 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845017.553502:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d90a80 with x1875739641378304 ext(1359872->2047999) 00002000:00010000:1.0:1788845017.553510:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d90a80 00000100:00100000:0.0:1788845017.568596:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.577305:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641378432:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 8408us (8711us total) trans 4294967364 rc 0/0 00000100:00100000:1.0:1788845017.577325:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 177 00002000:00100000:1.0:1788845017.577342:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (2019328->2045177) req@ffff94a034a0df80 x1875739641378432/t4294967364(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:123/0 lens 488/448 e 0 to 0 dl 1788845028 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845017.577355:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0df80 with x1875739641378432 ext(2019328->2045177) 00002000:00010000:1.0:1788845017.577363:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0df80 00000100:00100000:3.0:1788845017.586216:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641378560 00000100:00100000:3.0:1788845017.586226:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641378560 00000100:00100000:3.0:1788845017.586301:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 375 00000100:00100000:3.0:1788845017.586310:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641378560:12345-192.168.201.29@tcp:16:fsx.0 00010000:00010000:1.0:1788845017.682309:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845017.746738:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.764451:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.778817:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.778859:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.779029:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641378944 00000100:00100000:1.0:1788845017.779044:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641378944 00000100:00100000:1.0:1788845017.779131:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 605 00000100:00100000:1.0:1788845017.779142:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641378944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845017.779164:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845017.779187:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3a00/0xb2187301287abbf0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a76c8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.779224:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845017.779266:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb6680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660170112:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845017.779404:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788845017.783123:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845016 -> 1788845017 00000100:00100000:1.0:1788845017.783170:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb6680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660170112:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845017.783201:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845017.783220:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3a00/0xb2187301287abbf0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76c8 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.783243:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845017.783246:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845017.783251:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f027e3a00/0xb2187301287abbf0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76c8 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.783266:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3a00, rc 0) 00000100:00100000:1.0:1788845017.783326:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641378944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4185us (4465us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845017.783335:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 605 00000100:00100000:0.0:1788845017.789311:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.796289:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.796387:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641379328 00000100:00100000:1.0:1788845017.796397:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641379328 00010000:00010000:1.0:1788845017.796444:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 14 pid: 5839 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845017.796487:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 219 00000100:00100000:1.0:1788845017.796497:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641379328:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845017.796512:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845017.796535:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 14 pid: 5839 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.796554:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 14 pid: 5839 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.796593:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abbdb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76ba expref: 14 pid: 5839 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.796608:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845017.796614:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abbfe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76d6 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.796625:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abbfe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76d6 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.796646:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abbfe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76d6 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.796683:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abbfe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76d6 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845017.796698:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660170496:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845017.796780:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845017.801566:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.806673:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660170496:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845017.806705:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845017.806777:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641379328:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10281us (10499us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845017.806788:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 219 00000100:00100000:3.0:1788845017.809028:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.814498:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.828934:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.839451:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.839589:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641379968 00000100:00100000:0.0:1788845017.839599:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641379968 00000100:00100000:0.0:1788845017.839700:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 607 00000100:00100000:0.0:1788845017.839711:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641379968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845017.839732:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845017.839753:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ac00/0xb2187301287abc05 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a76dd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.839843:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abc0c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.839858:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abc0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.839871:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abc0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845017.839928:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2045178 00000004:00200000:0.0:1788845017.839933:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845017.839952:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abc0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.839977:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ac00/0xb2187301287abc05 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76dd expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.840009:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abc0c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76dd expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.840017:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845017.840020:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845017.840023:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a400, rc 0) 00000100:00100000:0.0:1788845017.840085:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641379968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 376us (645us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845017.840093:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 607 00000100:00100000:3.0:1788845017.842598:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.842843:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641380096 00000100:00100000:1.0:1788845017.842868:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641380096 00000100:00100000:1.0:1788845017.842979:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 377 00000100:00100000:1.0:1788845017.843000:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641380096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845017.843017:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845017.843036:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a1612e9a/0xb2187301287abc13 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a76e4 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845017.843095:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660170624:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845017.843211:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845017.847616:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660170624:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845017.847678:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a1612e9a/0xb2187301287abc13 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76e4 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.847707:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a1612e9a/0xb2187301287abc13 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76e4 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845017.847721:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e3000, rc 0) 00000100:00100000:1.0:1788845017.847820:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641380096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4831us (5229us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845017.847829:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 377 00000100:00100000:3.0:1788845017.858551:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641380224 00000100:00100000:3.0:1788845017.858562:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641380224 00000100:00100000:3.0:1788845017.858621:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 378 00000100:00100000:3.0:1788845017.858636:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641380224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.858655:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.858672:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287abc1a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a76eb expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.858725:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660170752:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.862613:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845017.881591:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641380352 00000100:00100000:3.0:1788845017.881600:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641380352 00000100:00100000:3.0:1788845017.881735:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 379 00000100:00100000:3.0:1788845017.881746:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641380352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.881764:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.881781:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287abc21 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a76f2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.881836:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660170880:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.881938:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845017.884309:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660170880:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845017.884353:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287abc21 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76f2 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.884377:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287abc21 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76f2 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.884702:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:0.0:1788845017.884765:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641380352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3021us (3260us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845017.884774:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 379 00000100:00100000:3.0:1788845017.888306:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.889266:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641380480 00000100:00100000:1.0:1788845017.889278:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641380480 00000100:00100000:1.0:1788845017.889572:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 380 00000100:00100000:1.0:1788845017.889584:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641380480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845017.889603:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845017.889641:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f535bca9/0xb2187301287abc28 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a76f9 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845017.889701:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660171008:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845017.889787:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845017.893232:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660171008:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845017.893281:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f535bca9/0xb2187301287abc28 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76f9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.893346:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f535bca9/0xb2187301287abc28 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76f9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.893360:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f027e2e00, rc 0) 00000100:00100000:0.0:1788845017.893638:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641380480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4055us (5350us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845017.893647:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 380 00000100:00100000:0.0:1788845017.897970:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.898139:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641380608 00000100:00100000:3.0:1788845017.898149:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641380608 00000100:00100000:3.0:1788845017.898244:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 381 00000100:00100000:3.0:1788845017.898266:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641380608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.898286:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.898317:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287abc2f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7700 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.898380:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660171136:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.898630:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845017.901718:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660171136:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845017.901776:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287abc2f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7700 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.901806:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287abc2f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7700 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.901824:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845017.901936:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641380608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3680us (4050us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845017.901945:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 381 00000100:00100000:0.0:1788845017.907093:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.923933:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.938991:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.946952:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845017.947715:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641381120 00000100:00100000:1.0:1788845017.947725:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641381120 00002000:00100000:1.0:1788845017.947775:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1359872->2047999) req@ffff94a035be6a00 x1875739641381120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:123/0 lens 488/0 e 0 to 0 dl 1788845028 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845017.947786:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be6a00 with x1875739641381120 ext(1359872->2047999) 00002000:00010000:1.0:1788845017.947791:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be6a00 00000100:00100000:1.0:1788845017.947831:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 178 00000100:00100000:1.0:1788845017.947840:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641381120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845017.974951:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.975012:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845017.975155:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641381376 00000100:00100000:0.0:1788845017.975165:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641381376 00000100:00100000:0.0:1788845017.975223:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 384 00000100:00100000:0.0:1788845017.975234:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641381376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845017.975251:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845017.975269:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abc44 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a771c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845017.975327:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a17b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660171648:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845017.975409:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00002000:00010000:0.0:1788845017.979342:0:6685:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845017 -> 1788845018 00002000:00010000:0.0:1788845017.979352:0:6685:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845017 -> 1788845018 00000100:00100000:0.0:1788845017.979379:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a17b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660171648:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845017.979429:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abc44 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a771c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.979466:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287abc44 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a771c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845017.979480:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a200, rc 0) 00000100:00100000:0.0:1788845017.979544:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641381376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4312us (4531us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845017.979553:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 384 00000100:00100000:1.0:1788845017.984945:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845017.985037:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641381504 00000100:00100000:3.0:1788845017.985059:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641381504 00000100:00100000:3.0:1788845017.985121:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 385 00000100:00100000:3.0:1788845017.985131:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641381504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845017.985149:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845017.985168:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287abc52 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7723 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845017.985230:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660171776:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845017.985310:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845017.988545:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660171776:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845017.988587:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287abc52 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7723 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.988611:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287abc52 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7723 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845017.989458:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845018.002801:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.002922:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641381632 00000100:00100000:0.0:1788845018.002939:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641381632 00002000:00100000:0.0:1788845018.003023:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb4e00 x1875739641381632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.003035:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb4e00 with x1875739641381632 00000100:00100000:0.0:1788845018.003087:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 329 00000100:00100000:0.0:1788845018.003100:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641381632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845018.019682:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641381632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 16573us (16880us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1788845018.019697:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 329 00002000:00100000:0.0:1788845018.019710:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb4e00 x1875739641381632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/440 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.019721:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb4e00 with x1875739641381632 00000100:00100000:0.0:1788845018.033236:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.033342:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641381760 00000100:00100000:1.0:1788845018.033353:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641381760 00002000:00100000:1.0:1788845018.033526:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1658880->1683455) req@ffff94a034a15180 x1875739641381760/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.033538:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a15180 with x1875739641381760 ext(1658880->1683455) 00002000:00010000:1.0:1788845018.033545:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a15180 00000100:00100000:1.0:1788845018.033579:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 179 00000100:00100000:1.0:1788845018.033589:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641381760:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845018.054786:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.061154:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.061246:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641382016 00000100:00100000:3.0:1788845018.061256:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641382016 00000100:00100000:3.0:1788845018.061318:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 386 00000100:00100000:3.0:1788845018.061329:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641382016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.061346:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.061363:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287abc59 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a772a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.061429:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660171904:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845018.061532:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.072947:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660171904:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845018.073003:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287abc59 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a772a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.073029:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287abc59 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a772a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.073045:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:3.0:1788845018.073116:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641382016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11789us (11971us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845018.073125:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 386 00000100:00100000:1.0:1788845018.078494:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.087247:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.087672:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641382400 00000100:00100000:1.0:1788845018.087683:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641382400 00000100:00100000:1.0:1788845018.087743:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 609 00000100:00100000:1.0:1788845018.087754:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641382400:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845018.087810:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845018.087838:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abc0c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76dd expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.087857:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845018.087861:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845018.088113:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287abc67 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.088128:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287abbe9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76c1 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.088136:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287abc67 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.088169:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287abbe9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76c1 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.088626:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287abbe9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76c1 expref: 13 pid: 5841 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.088646:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660172160:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845018.088827:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845018.093297:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845018.093998:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287abbe9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a76c1 expref: 13 pid: 5841 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.094021:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660172160:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845018.094050:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287abc67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.094058:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287abc67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.094065:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287abc67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845018.094189:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967555 00000004:00010000:0.0:1788845018.094241:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788845018.094244:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845017 -> 1788845018 00000004:00010000:0.0:1788845018.094247:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845017 -> 1788845016 00000004:00010000:0.0:1788845018.094249:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845017 -> 1788845018 00000004:00010000:0.0:1788845018.094251:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:0.0:1788845018.094408:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000922afef0 reply state 00000000d4fbab5c transno 4294967556 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287abc67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.094417:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845018.094481:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641382400:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6729us (7245us total) trans 4294967556 rc 0/0 00000100:00100000:0.0:1788845018.094488:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 609 00000020:00080000:0.0:1788845018.094492:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845018.097098:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.097330:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641382656 00000100:00100000:3.0:1788845018.097340:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641382656 00000100:00100000:3.0:1788845018.097392:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 610 00000100:00100000:3.0:1788845018.097401:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641382656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.097424:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.097426:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845018.097457:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abbfe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a76d6 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.097467:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.097471:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845018.097482:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287abc6e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7738 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.097515:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abc4b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7715 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.097524:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287abc6e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7738 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.097555:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abc4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7715 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.097572:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abc4b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7715 expref: 12 pid: 5840 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.097585:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660172288:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845018.104069:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.104346:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660172288:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845018.104378:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287abc6e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7738 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.104393:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.104395:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845018.104398:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287abc6e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7738 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.104406:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:3.0:1788845018.104483:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641382656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7084us (7394us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845018.104505:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 610 00000100:00100000:3.0:1788845018.104538:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641382784 00000100:00100000:3.0:1788845018.104543:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641382784 00010000:00010000:3.0:1788845018.104562:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abc4b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7715 expref: 13 pid: 5840 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.104585:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 222 00000100:00100000:3.0:1788845018.104591:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082fd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641382784:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845018.104600:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845018.104628:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abc4b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7715 expref: 13 pid: 5840 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.104640:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abc4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7715 expref: 13 pid: 5840 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.104657:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abc4b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7715 expref: 13 pid: 5840 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.104665:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.104668:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287abc6e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7738 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.104675:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287abc6e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7738 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.104680:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287abc6e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7738 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.104712:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287abc6e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7738 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.104725:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660172416:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845018.104770:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845018.108076:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.108194:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660172416:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845018.108214:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845018.108275:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082fd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641382784:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3683us (4216us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845018.108283:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 222 00000100:00100000:3.0:1788845018.112607:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.112988:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641383040 00000100:00100000:1.0:1788845018.112997:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641383040 00002000:00100000:1.0:1788845018.113038:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (2048000->2097151) req@ffff94a035be7480 x1875739641383040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.113047:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be7480 with x1875739641383040 ext(2048000->2097151) 00002000:00010000:1.0:1788845018.113051:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be7480 00000100:00100000:1.0:1788845018.113069:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 182 00000100:00100000:1.0:1788845018.113079:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2368:x1875739641383040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.115438:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2368:x1875739641383040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 2352us (2829us total) trans 0 rc 49152/49152 00000100:00100000:1.0:1788845018.115448:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 182 00002000:00100000:1.0:1788845018.115457:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (2048000->2097151) req@ffff94a035be7480 x1875739641383040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/440 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 49152/49152 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.115464:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be7480 with x1875739641383040 ext(2048000->2097151) 00002000:00010000:1.0:1788845018.115468:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be7480 00000100:00100000:3.0:1788845018.122095:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.122210:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641383168 00000100:00100000:1.0:1788845018.122221:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641383168 00002000:00100000:1.0:1788845018.122270:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (2060288->2095915) req@ffff94a035be7100 x1875739641383168/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.122280:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be7100 with x1875739641383168 ext(2060288->2095915) 00002000:00010000:1.0:1788845018.122285:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be7100 00000100:00100000:1.0:1788845018.122399:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 183 00000100:00100000:1.0:1788845018.122410:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641383168:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845018.129716:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.131079:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641383296 00000100:00100000:0.0:1788845018.131094:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641383296 00000100:00100000:0.0:1788845018.131185:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 388 00000100:00100000:0.0:1788845018.131194:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641383296:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845018.165234:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967368 is committed 00000100:00100000:0.0:1788845018.165427:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641383296:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 34228us (35715us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845018.165441:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 388 00000100:00100000:3.0:1788845018.168975:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.169281:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641383424 00000100:00100000:0.0:1788845018.169300:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641383424 00000100:00100000:0.0:1788845018.169360:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 611 00000100:00100000:0.0:1788845018.169370:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641383424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.169391:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.169412:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abc75 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a773f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.169506:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ba00/0xb2187301287abc7c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.169521:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ba00/0xb2187301287abc7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.169532:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ba00/0xb2187301287abc7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845018.169558:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2095916 00000004:00200000:0.0:1788845018.169562:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845018.169581:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ba00/0xb2187301287abc7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.169605:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abc75 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a773f expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.169657:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ba00/0xb2187301287abc7c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a773f expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.169665:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845018.169668:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845018.169671:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251ba00, rc 0) 00000100:00100000:0.0:1788845018.169745:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641383424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 376us (779us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845018.169752:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 611 00000100:00100000:3.0:1788845018.177528:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.184629:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.190611:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.200240:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.203832:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.204539:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641384064 00000100:00100000:0.0:1788845018.204555:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641384064 00000100:00100000:0.0:1788845018.204609:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 613 00000100:00100000:0.0:1788845018.204618:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641384064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.204637:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.204656:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abc8a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a774d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.204745:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abc91 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.204760:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abc91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.204771:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abc91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845018.204795:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2095916 00000004:00200000:0.0:1788845018.204810:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845018.204829:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abc91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.204851:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abc8a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a774d expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.204868:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abc91 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a774d expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.204876:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845018.204879:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845018.205121:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a600, rc 0) 00000100:00100000:0.0:1788845018.205191:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641384064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 574us (1367us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845018.205197:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 613 00000100:00100000:1.0:1788845018.209791:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.209869:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641384192 00000100:00100000:0.0:1788845018.209878:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641384192 00000100:00100000:0.0:1788845018.209956:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 389 00000100:00100000:0.0:1788845018.209965:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641384192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.209991:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.210008:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abc98 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7754 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.210061:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660172800:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845018.210138:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00002000:00010000:0.0:1788845018.213912:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845017 -> 1788845018 00002000:00010000:0.0:1788845018.213922:0:6682:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 251 -> 373 00000100:00100000:0.0:1788845018.213962:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660172800:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845018.214015:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abc98 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7754 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.214040:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abc98 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7754 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.214052:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251aa00, rc 0) 00000100:00100000:0.0:1788845018.214108:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641384192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4134us (4316us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845018.214117:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 389 00000100:00100000:1.0:1788845018.217642:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.217785:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641384320 00000100:00100000:3.0:1788845018.217795:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641384320 00000100:00100000:3.0:1788845018.217853:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 390 00000100:00100000:3.0:1788845018.217862:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641384320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.217880:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.217936:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abc9f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a775b expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.218028:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660172928:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845018.218117:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.221121:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660172928:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845018.221166:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abc9f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a775b expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.221192:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abc9f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a775b expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.221205:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845018.222467:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641384320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4017us (4247us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845018.222481:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 390 00000100:00100000:1.0:1788845018.225204:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.225298:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641384576 00000100:00100000:3.0:1788845018.225308:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641384576 00000100:00100000:3.0:1788845018.225367:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 614 00000100:00100000:3.0:1788845018.225378:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641384576:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845018.225430:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845018.225454:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287abc91 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a774d expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.225472:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.225477:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845018.225706:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.225719:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ba00/0xb2187301287abc7c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a773f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.225728:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.225761:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ba00/0xb2187301287abc7c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a773f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.225779:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ba00/0xb2187301287abc7c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a773f expref: 19 pid: 5839 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.225793:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660173056:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845018.225872:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.229534:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660173056:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845018.229563:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.231239:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845018.232287:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.232298:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845018.232414:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967557 00000004:00010000:0.0:1788845018.232467:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788845018.232471:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845018 -> 1788845018 00000004:00010000:0.0:1788845018.232473:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845016 -> 1788845016 00000004:00010000:0.0:1788845018.232476:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845018 -> 1788845018 00000004:00010000:0.0:1788845018.232478:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:0.0:1788845018.232485:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000890ad63 reply state 000000003976c711 transno 4294967558 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.232493:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845018.232593:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641384576:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 7216us (7395us total) trans 4294967558 rc 0/0 00000100:00100000:0.0:1788845018.232600:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 614 00000020:00080000:0.0:1788845018.232604:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788845018.241128:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845018.243685:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.243712:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.243720:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.243726:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d15200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845018.243731:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.243738:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.243755:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845018.243759:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845018.243763:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.243770:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287abca6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.249047:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.260855:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.261081:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641385088 00000100:00100000:0.0:1788845018.261091:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641385088 00002000:00010000:0.0:1788845018.261139:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1812821->18446744073709551615) 00002000:00100000:0.0:1788845018.261144:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f0a80 with x1875739641385088 ext(1812821->18446744073709551615) 00002000:00010000:0.0:1788845018.261170:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f0a80. 00000100:00100000:0.0:1788845018.261207:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 184 00000100:00100000:0.0:1788845018.261218:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641385088:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845018.261640:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2095916 -> 1812821 00002000:00010000:0.0:1788845018.261644:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845018 -> 1788845018 00002000:00010000:0.0:1788845018.261647:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845018 -> 1788845017 00002000:00010000:0.0:1788845018.261649:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845018 -> 1788845018 00002000:00010000:0.0:1788845018.261651:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 373 -> 373 00000100:00100000:0.0:1788845018.261708:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641385088:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 492us (862us total) trans 4294967369 rc 0/0 00000100:00100000:0.0:1788845018.261717:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 184 00002000:00010000:0.0:1788845018.261721:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1812821->18446744073709551615) 00002000:00100000:0.0:1788845018.261723:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f0a80 with x1875739641385088 ext(1812821->18446744073709551615) 00002000:00010000:0.0:1788845018.261728:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f0a80. 00000100:00100000:1.0:1788845018.264999:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.265456:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641385216 00000100:00100000:0.0:1788845018.265467:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641385216 00000100:00100000:0.0:1788845018.265529:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 615 00000100:00100000:0.0:1788845018.265539:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641385216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.265560:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.265583:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abcb4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7769 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.265680:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.265755:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.265782:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845018.265809:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1812821 00000004:00200000:0.0:1788845018.265814:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845018.265832:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.265857:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abcb4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.265876:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.266157:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845018.266160:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845018.266164:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b800, rc 0) 00000100:00100000:0.0:1788845018.266229:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641385216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 691us (1249us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845018.266237:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 615 00000100:00100000:1.0:1788845018.270645:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.270703:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.270771:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641385472 00000100:00100000:0.0:1788845018.270780:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641385472 00000100:00100000:0.0:1788845018.270843:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 392 00000100:00100000:0.0:1788845018.270853:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641385472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.270872:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.270911:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abcc2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7777 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.270995:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660173440:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845018.271061:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00002000:00010000:0.0:1788845018.274116:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845017 -> 1788845018 00000100:00100000:0.0:1788845018.274148:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660173440:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845018.274185:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abcc2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7777 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.274210:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abcc2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7777 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.274223:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251aa00, rc 0) 00000100:00100000:0.0:1788845018.274283:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641385472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3431us (3586us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845018.274300:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 392 00000100:00100000:1.0:1788845018.284245:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.291415:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.291557:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641385728 00000100:00100000:0.0:1788845018.291566:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641385728 00000100:00100000:0.0:1788845018.291630:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 393 00000100:00100000:0.0:1788845018.291640:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641385728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.291659:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.291678:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abcd7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7785 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.291749:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660173824:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845018.291848:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845018.300495:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660173824:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845018.300535:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abcd7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7785 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.300557:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abcd7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7785 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.300570:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251aa00, rc 0) 00000100:00100000:0.0:1788845018.300629:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641385728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8990us (9223us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845018.300637:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 393 00000100:00100000:1.0:1788845018.309741:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.309937:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641385856 00000100:00100000:0.0:1788845018.309946:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641385856 00000100:00100000:0.0:1788845018.310017:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 618 00000100:00100000:0.0:1788845018.310028:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641385856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.310059:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.310081:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abcde lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a778c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.310183:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abce5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.310198:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.310209:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845018.310251:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1812821 00000004:00200000:0.0:1788845018.310255:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845018.310274:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.310306:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abcde lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a778c expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.310324:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abce5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a778c expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.310333:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845018.310336:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845018.310340:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251be00, rc 0) 00000100:00100000:0.0:1788845018.310410:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641385856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 384us (680us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845018.310417:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 618 00000100:00100000:1.0:1788845018.327056:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.327124:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.327221:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641386112 00000100:00100000:0.0:1788845018.327230:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641386112 00002000:00100000:0.0:1788845018.327277:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3800 x1875739641386112/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 504/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.327288:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3800 with x1875739641386112 00000100:00100000:0.0:1788845018.327337:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 332 00000100:00100000:0.0:1788845018.327346:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641386112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845018.388431:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641386112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 61073us (61293us total) trans 0 rc 1011712/1011712 00000100:00100000:0.0:1788845018.388461:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 332 00002000:00100000:0.0:1788845018.388476:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3800 x1875739641386112/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 504/440 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 1011712/1011712 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.388488:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3800 with x1875739641386112 00000100:00100000:0.0:1788845018.401142:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641386240 00000100:00100000:0.0:1788845018.401151:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641386240 00000100:00100000:0.0:1788845018.401219:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 619 00000100:00100000:0.0:1788845018.401231:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2367:x1875739641386240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.401253:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.401273:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abcec lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7793 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.401309:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.401346:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a14700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660173952:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845018.401443:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.401599:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641386368 00000100:00100000:3.0:1788845018.401603:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641386368 00000100:00100000:3.0:1788845018.401649:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 394 00000100:00100000:3.0:1788845018.401657:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641386368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.401666:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.401676:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abcf3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a779a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.401741:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660174080:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845018.401769:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845018.405659:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a14700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660173952:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845018.405688:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845018.405718:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abcec lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7793 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.405738:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845018.405741:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845018.405747:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abcec lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7793 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.405778:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251aa00, rc 0) 00000100:00100000:0.0:1788845018.405849:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2367:x1875739641386240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4620us (4847us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845018.405857:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 619 00000100:00100000:3.0:1788845018.406726:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660174080:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845018.406755:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abcf3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a779a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.406776:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abcf3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a779a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.406788:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845018.406917:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641386368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5227us (5834us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845018.406925:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 394 00000100:00100000:1.0:1788845018.413154:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.413539:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641386496 00000100:00100000:3.0:1788845018.413548:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641386496 00000100:00100000:3.0:1788845018.413789:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 395 00000100:00100000:3.0:1788845018.413801:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641386496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.413820:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.413838:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77a1 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.413870:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287abcad lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7762 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.413934:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 6) 00010000:00010000:3.0:1788845018.413938:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 6) 00010000:00010000:3.0:1788845018.413944:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1327104->1331199] (req 1327104->1331199) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.413993:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287abcad lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7762 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.414012:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287abcad lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7762 expref: 6 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.414025:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660174208:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845018.414123:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.418014:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.419533:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641386624 00000100:00100000:1.0:1788845018.419542:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641386624 00010000:00010000:1.0:1788845018.419578:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287abcad lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7762 expref: 7 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.419623:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 226 00000100:00100000:1.0:1788845018.419633:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641386624:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845018.419646:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845018.419676:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287abcad lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7762 expref: 7 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.419705:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287abcad lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7762 expref: 7 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.419727:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287abcad lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7762 expref: 7 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.419741:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 6) 00010000:00010000:1.0:1788845018.419745:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 6) 00010000:00010000:1.0:1788845018.419753:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.419763:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.419802:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.419815:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660174336:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845018.419902:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.424946:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.425084:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035446d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660174336:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845018.425123:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845018.425174:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641386624:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5541us (7170us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845018.425198:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 226 00000100:00100000:3.0:1788845018.431249:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.436064:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.436276:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641387008 00000100:00100000:1.0:1788845018.436286:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641387008 00002000:00100000:1.0:1788845018.436337:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1335296->1339391) req@ffff94a034a14700 x1875739641387008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.436348:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a14700 with x1875739641387008 ext(1335296->1339391) 00002000:00010000:1.0:1788845018.436353:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a14700 00000100:00100000:1.0:1788845018.436392:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 188 00000100:00100000:1.0:1788845018.436401:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.436642:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 241us (588us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.436651:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 188 00002000:00100000:1.0:1788845018.436657:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1335296->1339391) req@ffff94a034a14700 x1875739641387008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.436666:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a14700 with x1875739641387008 ext(1335296->1339391) 00002000:00010000:1.0:1788845018.436669:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a14700 00000100:00100000:0.0:1788845018.440149:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.440367:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641387136 00000100:00100000:1.0:1788845018.440377:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641387136 00002000:00100000:1.0:1788845018.440427:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1339392->1343487) req@ffff94a034a16d80 x1875739641387136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.440439:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a16d80 with x1875739641387136 ext(1339392->1343487) 00002000:00010000:1.0:1788845018.440445:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a16d80 00000100:00100000:1.0:1788845018.440477:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 189 00000100:00100000:1.0:1788845018.440486:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.440743:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 257us (603us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.440750:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 189 00002000:00100000:1.0:1788845018.440756:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1339392->1343487) req@ffff94a034a16d80 x1875739641387136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.440763:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a16d80 with x1875739641387136 ext(1339392->1343487) 00002000:00010000:1.0:1788845018.440767:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a16d80 00000100:00100000:0.0:1788845018.444768:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.448865:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.448987:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641387392 00000100:00100000:3.0:1788845018.448996:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641387392 00002000:00100000:3.0:1788845018.449039:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1347584->1351679) req@ffff94a034a14a80 x1875739641387392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.449050:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a14a80 with x1875739641387392 ext(1347584->1351679) 00002000:00010000:3.0:1788845018.449055:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a14a80 00000100:00100000:3.0:1788845018.449211:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 191 00000100:00100000:3.0:1788845018.449220:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845018.449601:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 381us (744us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845018.449609:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 191 00002000:00100000:3.0:1788845018.449624:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1347584->1351679) req@ffff94a034a14a80 x1875739641387392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.449632:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a14a80 with x1875739641387392 ext(1347584->1351679) 00002000:00010000:3.0:1788845018.449635:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a14a80 00000100:00100000:0.0:1788845018.455503:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.455686:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641387520 00000100:00100000:1.0:1788845018.455765:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641387520 00002000:00100000:1.0:1788845018.455817:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1351680->1355775) req@ffff94a034a14000 x1875739641387520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.455829:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a14000 with x1875739641387520 ext(1351680->1355775) 00002000:00010000:1.0:1788845018.455834:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a14000 00000100:00100000:1.0:1788845018.455952:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 192 00000100:00100000:1.0:1788845018.455963:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.456269:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 306us (774us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.456276:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 192 00002000:00100000:1.0:1788845018.456283:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1351680->1355775) req@ffff94a034a14000 x1875739641387520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.456291:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a14000 with x1875739641387520 ext(1351680->1355775) 00002000:00010000:1.0:1788845018.456294:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a14000 00000100:00100000:0.0:1788845018.460230:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.460329:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641387648 00000100:00100000:3.0:1788845018.460339:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641387648 00002000:00100000:3.0:1788845018.460392:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1355776->1359871) req@ffff94a034a16a00 x1875739641387648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.460404:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a16a00 with x1875739641387648 ext(1355776->1359871) 00002000:00010000:3.0:1788845018.460410:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a16a00 00000100:00100000:3.0:1788845018.460496:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 193 00000100:00100000:3.0:1788845018.460507:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845018.460782:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641387648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 275us (560us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845018.460789:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 193 00002000:00100000:3.0:1788845018.460797:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1355776->1359871) req@ffff94a034a16a00 x1875739641387648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.460805:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a16a00 with x1875739641387648 ext(1355776->1359871) 00002000:00010000:3.0:1788845018.460809:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a16a00 00000100:00100000:0.0:1788845018.465585:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.465700:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641387776 00000100:00100000:3.0:1788845018.465726:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641387776 00000100:00100000:3.0:1788845018.465790:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 620 00000100:00100000:3.0:1788845018.465801:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641387776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.465822:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.465844:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abd01 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77a8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.465879:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.465954:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660174464:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845018.466029:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.472553:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660174464:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845018.472586:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845018.472608:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abd01 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a8 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.472644:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.472647:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845018.472653:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abd01 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a8 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.472670:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845018.472758:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641387776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6959us (7183us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845018.472767:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 620 00000100:00100000:0.0:1788845018.480328:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.486163:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660174592:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845018.486212:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845018.486235:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abd08 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77af expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.487434:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845018.487442:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845018.487450:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abd08 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77af expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.487471:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:1.0:1788845018.487596:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641387904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7085us (7277us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845018.487606:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 621 00000100:00100000:0.0:1788845018.496227:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.496351:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641388032 00000100:00100000:1.0:1788845018.496361:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641388032 00000100:00100000:1.0:1788845018.496421:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 396 00000100:00100000:1.0:1788845018.496431:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641388032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845018.496452:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845018.496470:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287abd0f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77b6 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.496497:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 6) 00010000:00010000:1.0:1788845018.496502:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 6) 00010000:00010000:1.0:1788845018.496513:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287abd0f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77b6 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.496531:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287abd0f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77b6 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.496560:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb400, rc 0) 00000100:00100000:1.0:1788845018.496631:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641388032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 201us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845018.496638:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 396 00000100:00100000:1.0:1788845018.502810:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.503021:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641388160 00000100:00100000:0.0:1788845018.503032:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641388160 00002000:00100000:0.0:1788845018.503084:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1601535) req@ffff94a035446d80 x1875739641388160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.503106:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035446d80 with x1875739641388160 ext(1597440->1601535) 00002000:00010000:0.0:1788845018.503112:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035446d80 00000100:00100000:0.0:1788845018.503147:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 194 00000100:00100000:0.0:1788845018.503158:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035446d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641388160:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845018.503421:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641388160:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 264us (621us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845018.503430:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 194 00002000:00100000:0.0:1788845018.503453:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1601535) req@ffff94a035446d80 x1875739641388160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.503461:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035446d80 with x1875739641388160 ext(1597440->1601535) 00002000:00010000:0.0:1788845018.503464:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035446d80 00000100:00100000:3.0:1788845018.506871:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641388288 00000100:00100000:3.0:1788845018.506938:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641388288 00002000:00100000:3.0:1788845018.506995:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1601536->1605631) req@ffff94a019d93800 x1875739641388288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.507006:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d93800 with x1875739641388288 ext(1601536->1605631) 00002000:00010000:3.0:1788845018.507012:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d93800 00000100:00100000:3.0:1788845018.507037:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 195 00000100:00100000:3.0:1788845018.507046:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641388288:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845018.507300:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641388288:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (690us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845018.507308:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 195 00002000:00100000:3.0:1788845018.507315:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1601536->1605631) req@ffff94a019d93800 x1875739641388288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.507324:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d93800 with x1875739641388288 ext(1601536->1605631) 00002000:00010000:3.0:1788845018.507327:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d93800 00000100:00100000:3.0:1788845018.516009:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.520645:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.520802:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641388544 00000100:00100000:0.0:1788845018.520813:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641388544 00002000:00100000:0.0:1788845018.520860:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1609728->1613823) req@ffff94a035ef8e00 x1875739641388544/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.520872:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035ef8e00 with x1875739641388544 ext(1609728->1613823) 00002000:00010000:0.0:1788845018.520877:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035ef8e00 00000100:00100000:0.0:1788845018.520954:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 197 00000100:00100000:0.0:1788845018.520963:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641388544:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845018.521224:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641388544:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (589us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845018.521232:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 197 00002000:00100000:0.0:1788845018.521240:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1609728->1613823) req@ffff94a035ef8e00 x1875739641388544/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.521248:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035ef8e00 with x1875739641388544 ext(1609728->1613823) 00002000:00010000:0.0:1788845018.521252:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035ef8e00 00000100:00100000:0.0:1788845018.525788:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.526317:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641388672 00000100:00100000:3.0:1788845018.526327:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641388672 00002000:00100000:3.0:1788845018.526381:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1613824->1617919) req@ffff94a034a14380 x1875739641388672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.526393:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a14380 with x1875739641388672 ext(1613824->1617919) 00002000:00010000:3.0:1788845018.526399:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a14380 00000100:00100000:3.0:1788845018.526472:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 198 00000100:00100000:3.0:1788845018.526483:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641388672:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845018.526761:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641388672:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 278us (981us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845018.526775:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 198 00002000:00100000:3.0:1788845018.526783:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1613824->1617919) req@ffff94a034a14380 x1875739641388672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.526791:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a14380 with x1875739641388672 ext(1613824->1617919) 00002000:00010000:3.0:1788845018.526794:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a14380 00000100:00100000:3.0:1788845018.530673:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.535609:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.535792:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641388928 00000100:00100000:3.0:1788845018.535803:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641388928 00002000:00100000:3.0:1788845018.535855:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1622016->1626111) req@ffff94a035efad80 x1875739641388928/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.535866:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035efad80 with x1875739641388928 ext(1622016->1626111) 00002000:00010000:3.0:1788845018.535872:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035efad80 00000100:00100000:3.0:1788845018.535937:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 200 00000100:00100000:3.0:1788845018.535947:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641388928:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845018.540662:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.541576:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641389056 00000100:00100000:0.0:1788845018.541590:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641389056 00002000:00100000:0.0:1788845018.541830:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1626112->1630207) req@ffff94a0360e8380 x1875739641389056/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845018.541862:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360e8380 with x1875739641389056 ext(1626112->1630207) 00002000:00010000:0.0:1788845018.541867:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360e8380 00000100:00100000:1.0:1788845018.542092:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 201 00000100:00100000:1.0:1788845018.542115:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641389056:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.542394:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641389056:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (1739us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.542402:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 201 00002000:00100000:1.0:1788845018.542409:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1626112->1630207) req@ffff94a0360e8380 x1875739641389056/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.542417:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360e8380 with x1875739641389056 ext(1626112->1630207) 00002000:00010000:1.0:1788845018.542421:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360e8380 00000100:00100000:3.0:1788845018.546178:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.557337:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.557489:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641389312 00000100:00100000:1.0:1788845018.557500:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641389312 00002000:00100000:1.0:1788845018.557556:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1634304->1638399) req@ffff949f01ced880 x1875739641389312/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.557568:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01ced880 with x1875739641389312 ext(1634304->1638399) 00002000:00010000:1.0:1788845018.557573:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01ced880 00000100:00100000:1.0:1788845018.557608:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 203 00000100:00100000:1.0:1788845018.557644:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641389312:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.558066:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641389312:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 418us (732us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.558074:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 203 00002000:00100000:1.0:1788845018.558082:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1634304->1638399) req@ffff949f01ced880 x1875739641389312/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.558090:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01ced880 with x1875739641389312 ext(1634304->1638399) 00002000:00010000:1.0:1788845018.558102:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01ced880 00000100:00100000:3.0:1788845018.564360:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.564597:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641389440 00000100:00100000:1.0:1788845018.564608:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641389440 00002000:00100000:1.0:1788845018.564676:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1638400->1642495) req@ffff94a034f21180 x1875739641389440/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.564810:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f21180 with x1875739641389440 ext(1638400->1642495) 00002000:00010000:1.0:1788845018.564817:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f21180 00000100:00100000:1.0:1788845018.564857:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 204 00000100:00100000:1.0:1788845018.564869:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641389440:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.565398:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641389440:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 529us (1044us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.565406:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 204 00002000:00100000:1.0:1788845018.565414:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1638400->1642495) req@ffff94a034f21180 x1875739641389440/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.565423:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f21180 with x1875739641389440 ext(1638400->1642495) 00002000:00010000:1.0:1788845018.565426:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f21180 00000100:00100000:0.0:1788845018.574336:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.585115:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641389696 00000100:00100000:1.0:1788845018.585126:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641389696 00002000:00100000:1.0:1788845018.585175:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1646592->1650687) req@ffff94a019d91c00 x1875739641389696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.585185:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d91c00 with x1875739641389696 ext(1646592->1650687) 00002000:00010000:1.0:1788845018.585190:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d91c00 00000100:00100000:1.0:1788845018.585223:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 206 00000100:00100000:1.0:1788845018.585232:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641389696:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.585469:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641389696:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 237us (465us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.585476:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 206 00002000:00100000:1.0:1788845018.585483:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1646592->1650687) req@ffff94a019d91c00 x1875739641389696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.585491:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d91c00 with x1875739641389696 ext(1646592->1650687) 00002000:00010000:1.0:1788845018.585494:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d91c00 00000100:00100000:3.0:1788845018.589085:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.589197:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641389824 00000100:00100000:1.0:1788845018.589207:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641389824 00002000:00100000:1.0:1788845018.589258:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1650688->1654783) req@ffff94a034f23b80 x1875739641389824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.589269:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f23b80 with x1875739641389824 ext(1650688->1654783) 00002000:00010000:1.0:1788845018.589273:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f23b80 00000100:00100000:1.0:1788845018.589310:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 207 00000100:00100000:1.0:1788845018.589320:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641389824:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.589552:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641389824:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (476us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.589574:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 207 00002000:00100000:1.0:1788845018.589581:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1650688->1654783) req@ffff94a034f23b80 x1875739641389824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.589588:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f23b80 with x1875739641389824 ext(1650688->1654783) 00002000:00010000:1.0:1788845018.589591:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f23b80 00000100:00100000:0.0:1788845018.593683:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.599352:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.605029:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.605226:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641390208 00000100:00100000:1.0:1788845018.605236:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641390208 00002000:00100000:1.0:1788845018.605285:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1662976->1667071) req@ffff94a034f20000 x1875739641390208/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.605297:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f20000 with x1875739641390208 ext(1662976->1667071) 00002000:00010000:1.0:1788845018.605303:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f20000 00000100:00100000:1.0:1788845018.605341:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 210 00000100:00100000:1.0:1788845018.605351:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641390208:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.605564:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641390208:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 214us (544us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.605572:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 210 00002000:00100000:1.0:1788845018.605578:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1662976->1667071) req@ffff94a034f20000 x1875739641390208/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.605587:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f20000 with x1875739641390208 ext(1662976->1667071) 00002000:00010000:1.0:1788845018.605590:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f20000 00000100:00100000:0.0:1788845018.609674:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.616231:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.616386:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641390464 00000100:00100000:1.0:1788845018.616396:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641390464 00000100:00100000:1.0:1788845018.616455:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 623 00000100:00100000:1.0:1788845018.616465:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2368:x1875739641390464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845018.616498:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845018.616519:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287abd1d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77c4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.616555:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.616594:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660174848:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845018.616682:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845018.619989:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035efbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660174848:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845018.620023:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845018.620044:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287abd1d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77c4 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.620065:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845018.620068:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845018.620075:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287abd1d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77c4 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.620100:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba800, rc 0) 00000100:00100000:1.0:1788845018.620154:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2368:x1875739641390464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3690us (3932us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845018.620163:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 623 00000100:00100000:3.0:1788845018.623985:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.624212:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641390720 00000100:00100000:0.0:1788845018.624223:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641390720 00000100:00100000:0.0:1788845018.624303:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 624 00000100:00100000:0.0:1788845018.624314:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641390720:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845018.624368:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845018.624397:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abce5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a778c expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.624419:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845018.624425:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845018.624659:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.624674:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.624682:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.624728:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.624763:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 19 pid: 11036 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.624777:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660174976:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845018.624857:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845018.628787:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.629011:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660174976:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845018.629041:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.629677:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641390848 00000100:00100000:1.0:1788845018.629685:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641390848 00010000:00010000:1.0:1788845018.629872:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 20 pid: 11036 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.629947:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 227 00000100:00100000:1.0:1788845018.629957:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641390848:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845018.630119:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845018.630128:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 20 pid: 11036 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.630146:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 20 pid: 11036 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.630167:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abcbb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7769 expref: 20 pid: 11036 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.630182:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845018.630187:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.630200:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.630207:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.630237:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845018.630331:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.630339:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845018.630452:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967559 00000004:00010000:3.0:1788845018.630504:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845018.630507:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845018 -> 1788845018 00000004:00010000:3.0:1788845018.630510:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845018 -> 1788845016 00000004:00010000:3.0:1788845018.630512:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845018 -> 1788845018 00000004:00010000:3.0:1788845018.630514:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:3.0:1788845018.630522:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000683ba4d9 reply state 00000000c5e4345b transno 4294967560 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abd24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.630531:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845018.630650:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641390848:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 693us (1871us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845018.630658:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 227 00000100:00100000:3.0:1788845018.632416:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641390720:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8096us (8442us total) trans 4294967560 rc 0/0 00000100:00100000:3.0:1788845018.632428:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 624 00000020:00080000:3.0:1788845018.632435:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845018.644825:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.645551:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641390976 00000100:00100000:3.0:1788845018.645560:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641390976 00000100:00100000:3.0:1788845018.645707:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 397 00000100:00100000:3.0:1788845018.645719:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641390976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.645736:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.645754:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77cb expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.645793:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.645803:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287abd0f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77b6 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.645810:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 6) 00010000:00010000:3.0:1788845018.645814:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 6) 00010000:00010000:3.0:1788845018.645819:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.645852:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287abd0f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77b6 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.645869:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287abd0f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77b6 expref: 8 pid: 6685 timeout: 507 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.645900:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f23800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660175104:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845018.646000:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.646010:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 5 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.646033:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660175232:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845018.646056:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.649196:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.649287:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641391104 00000100:00100000:1.0:1788845018.649309:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641391104 00010000:00010000:1.0:1788845018.649349:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 6 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.649395:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 228 00000100:00100000:1.0:1788845018.649405:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a16680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641391104:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845018.649419:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845018.649442:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 6 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.649456:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 6 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.649482:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845018.649529:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a16680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641391104:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 126us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845018.649536:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 228 00010000:00010000:0.0:1788845018.649749:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287abcfa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1327104->1331199) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77a1 expref: 5 pid: 7983 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.649764:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660175232:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845018.653546:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845018.654636:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287abd0f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77b6 expref: 9 pid: 6685 timeout: 507 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.654655:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f23800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660175104:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845018.654684:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.654756:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:3.0:1788845018.654800:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641390976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9084us (9985us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845018.654808:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 397 00000100:00100000:0.0:1788845018.661222:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.661333:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641391360 00000100:00100000:1.0:1788845018.661343:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641391360 00002000:00010000:1.0:1788845018.661383:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1777329->18446744073709551615) 00002000:00100000:1.0:1788845018.661388:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb5180 with x1875739641391360 ext(1777329->18446744073709551615) 00002000:00010000:1.0:1788845018.661402:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb5180. 00000100:00100000:1.0:1788845018.661437:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 211 00000100:00100000:1.0:1788845018.661446:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641391360:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845018.661853:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1812821 -> 1777329 00002000:00010000:1.0:1788845018.661857:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845018 -> 1788845018 00002000:00010000:1.0:1788845018.661873:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845018 -> 1788845017 00002000:00010000:1.0:1788845018.661875:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845018 -> 1788845018 00002000:00010000:1.0:1788845018.661878:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 373 -> 373 00000100:00100000:1.0:1788845018.662059:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641391360:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 612us (845us total) trans 4294967370 rc 0/0 00000100:00100000:1.0:1788845018.662068:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 211 00002000:00010000:1.0:1788845018.662071:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1777329->18446744073709551615) 00002000:00100000:1.0:1788845018.662074:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb5180 with x1875739641391360 ext(1777329->18446744073709551615) 00002000:00010000:1.0:1788845018.662079:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb5180. 00000100:00100000:0.0:1788845018.665358:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.676403:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00002000:00100000:2.0:1788845018.676551:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0ca80 with x1875739641391616 ext(1404928->1409023) 00002000:00010000:2.0:1788845018.676570:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0ca80 00000100:00100000:2.0:1788845018.676608:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 212 00000100:00100000:2.0:1788845018.676628:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641391616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845018.676912:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641391616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 260us (517us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845018.676921:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 212 00002000:00100000:2.0:1788845018.676927:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1404928->1409023) req@ffff94a034a0ca80 x1875739641391616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.676936:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0ca80 with x1875739641391616 ext(1404928->1409023) 00002000:00010000:2.0:1788845018.676939:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0ca80 00000100:00100000:0.0:1788845018.683055:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.683165:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641391744 00000100:00100000:1.0:1788845018.683176:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641391744 00002000:00100000:1.0:1788845018.683238:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1433600->1437695) req@ffff94a034a0d180 x1875739641391744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.683249:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0d180 with x1875739641391744 ext(1433600->1437695) 00002000:00010000:1.0:1788845018.683254:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0d180 00000100:00100000:1.0:1788845018.683292:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 213 00000100:00100000:1.0:1788845018.683303:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641391744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.683557:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641391744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 255us (514us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.683565:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 213 00002000:00100000:1.0:1788845018.683572:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1433600->1437695) req@ffff94a034a0d180 x1875739641391744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.683581:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0d180 with x1875739641391744 ext(1433600->1437695) 00002000:00010000:1.0:1788845018.683584:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0d180 00000100:00100000:0.0:1788845018.697155:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.697350:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641391872 00000100:00100000:1.0:1788845018.697364:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641391872 00002000:00100000:1.0:1788845018.697416:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1572864->1576959) req@ffff94a034a0ed80 x1875739641391872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.697427:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0ed80 with x1875739641391872 ext(1572864->1576959) 00002000:00010000:1.0:1788845018.697433:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0ed80 00000100:00100000:1.0:1788845018.697475:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 214 00000100:00100000:1.0:1788845018.697486:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641391872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.697873:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641391872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 387us (727us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.698042:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 214 00002000:00100000:1.0:1788845018.698052:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1572864->1576959) req@ffff94a034a0ed80 x1875739641391872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.698060:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0ed80 with x1875739641391872 ext(1572864->1576959) 00002000:00010000:1.0:1788845018.698064:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0ed80 00000100:00100000:0.0:1788845018.704197:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.706921:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641392000 00000100:00100000:2.0:1788845018.706937:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641392000 00002000:00100000:2.0:1788845018.707057:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1576960->1581055) req@ffff94a034a0ce00 x1875739641392000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.707084:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0ce00 with x1875739641392000 ext(1576960->1581055) 00002000:00010000:2.0:1788845018.707100:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0ce00 00000100:00100000:2.0:1788845018.707137:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 215 00010000:00010000:0.0:1788845018.707141:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845018.707146:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845018.707654:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 505us (3463us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845018.707663:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 215 00002000:00100000:2.0:1788845018.707671:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1576960->1581055) req@ffff94a034a0ce00 x1875739641392000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.707679:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0ce00 with x1875739641392000 ext(1576960->1581055) 00002000:00010000:2.0:1788845018.707838:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0ce00 00000100:00100000:0.0:1788845018.716967:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.717030:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641392128 00000100:00100000:1.0:1788845018.717040:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641392128 00002000:00100000:1.0:1788845018.717109:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1581056->1585151) req@ffff94a034a0d880 x1875739641392128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.717120:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0d880 with x1875739641392128 ext(1581056->1585151) 00002000:00010000:1.0:1788845018.717126:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0d880 00000100:00100000:1.0:1788845018.717159:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 216 00000100:00100000:1.0:1788845018.717169:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641392128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.717491:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641392128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 322us (533us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.717513:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 216 00002000:00100000:1.0:1788845018.717520:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1581056->1585151) req@ffff94a034a0d880 x1875739641392128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.717528:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0d880 with x1875739641392128 ext(1581056->1585151) 00002000:00010000:1.0:1788845018.717531:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0d880 00000100:00100000:0.0:1788845018.721535:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.721645:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641392256 00000100:00100000:1.0:1788845018.721655:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641392256 00002000:00100000:1.0:1788845018.721705:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1585152->1589247) req@ffff94a034a0f100 x1875739641392256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.721716:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0f100 with x1875739641392256 ext(1585152->1589247) 00002000:00010000:1.0:1788845018.721721:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0f100 00000100:00100000:1.0:1788845018.721761:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 217 00000100:00100000:1.0:1788845018.721771:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641392256:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.722085:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641392256:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 313us (560us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.722101:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 217 00002000:00100000:1.0:1788845018.722108:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1585152->1589247) req@ffff94a034a0f100 x1875739641392256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.722115:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0f100 with x1875739641392256 ext(1585152->1589247) 00002000:00010000:1.0:1788845018.722119:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0f100 00000100:00100000:0.0:1788845018.740255:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.740392:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641392384 00000100:00100000:2.0:1788845018.740405:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641392384 00002000:00100000:2.0:1788845018.740460:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1589248->1593343) req@ffff94a034a0dc00 x1875739641392384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.740471:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0dc00 with x1875739641392384 ext(1589248->1593343) 00002000:00010000:2.0:1788845018.740477:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0dc00 00000100:00100000:2.0:1788845018.740518:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 218 00000100:00100000:2.0:1788845018.740528:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641392384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845018.740784:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641392384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 255us (538us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845018.740792:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 218 00002000:00100000:2.0:1788845018.740799:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1589248->1593343) req@ffff94a034a0dc00 x1875739641392384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.740808:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a0dc00 with x1875739641392384 ext(1589248->1593343) 00002000:00010000:2.0:1788845018.740812:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a0dc00 00000100:00100000:1.0:1788845018.744224:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.744337:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641392512 00000100:00100000:2.0:1788845018.744347:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641392512 00002000:00100000:2.0:1788845018.744402:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1597439) req@ffff94a035efbb80 x1875739641392512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.744414:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035efbb80 with x1875739641392512 ext(1593344->1597439) 00002000:00010000:2.0:1788845018.744420:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035efbb80 00000100:00100000:2.0:1788845018.744496:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 219 00000100:00100000:2.0:1788845018.744506:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845018.744769:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 263us (557us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845018.744777:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 219 00002000:00100000:2.0:1788845018.744785:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1597439) req@ffff94a035efbb80 x1875739641392512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.744793:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035efbb80 with x1875739641392512 ext(1593344->1597439) 00002000:00010000:2.0:1788845018.744796:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035efbb80 00000100:00100000:0.0:1788845018.753057:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.753167:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641392640 00000100:00100000:1.0:1788845018.753177:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641392640 00002000:00100000:1.0:1788845018.753235:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1601535) req@ffff949f024d6680 x1875739641392640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.753246:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d6680 with x1875739641392640 ext(1597440->1601535) 00002000:00010000:1.0:1788845018.753253:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d6680 00000100:00100000:1.0:1788845018.753340:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 220 00000100:00100000:1.0:1788845018.753350:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.753600:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 251us (552us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.753608:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 220 00002000:00100000:1.0:1788845018.753615:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1601535) req@ffff949f024d6680 x1875739641392640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.753623:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d6680 with x1875739641392640 ext(1597440->1601535) 00002000:00010000:1.0:1788845018.753626:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d6680 00000100:00100000:0.0:1788845018.766645:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.766844:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641392768 00000100:00100000:2.0:1788845018.766853:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641392768 00002000:00100000:2.0:1788845018.766928:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1601536->1605631) req@ffff949f024d7b80 x1875739641392768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.766939:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d7b80 with x1875739641392768 ext(1601536->1605631) 00002000:00010000:2.0:1788845018.766944:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d7b80 00000100:00100000:2.0:1788845018.766978:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 221 00000100:00100000:2.0:1788845018.766989:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845018.767227:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 240us (593us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845018.767235:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 221 00002000:00100000:2.0:1788845018.767243:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1601536->1605631) req@ffff949f024d7b80 x1875739641392768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.767252:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d7b80 with x1875739641392768 ext(1601536->1605631) 00002000:00010000:2.0:1788845018.767256:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d7b80 00000100:00100000:0.0:1788845018.776181:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.776443:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641392896 00000100:00100000:1.0:1788845018.776454:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641392896 00002000:00100000:1.0:1788845018.776509:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1605632->1609727) req@ffff949f024d4700 x1875739641392896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.776522:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d4700 with x1875739641392896 ext(1605632->1609727) 00002000:00010000:1.0:1788845018.776528:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d4700 00000100:00100000:1.0:1788845018.776564:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 222 00000100:00100000:1.0:1788845018.776573:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.776807:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641392896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (634us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.776814:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 222 00002000:00100000:1.0:1788845018.776827:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1605632->1609727) req@ffff949f024d4700 x1875739641392896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.776836:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d4700 with x1875739641392896 ext(1605632->1609727) 00002000:00010000:1.0:1788845018.776839:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d4700 00000100:00100000:0.0:1788845018.785991:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.786086:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641393024 00000100:00100000:1.0:1788845018.786106:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641393024 00002000:00100000:1.0:1788845018.786155:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1609728->1613823) req@ffff949f024d6d80 x1875739641393024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.786166:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d6d80 with x1875739641393024 ext(1609728->1613823) 00002000:00010000:1.0:1788845018.786171:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d6d80 00000100:00100000:1.0:1788845018.786218:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 223 00000100:00100000:1.0:1788845018.786227:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641393024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.786455:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641393024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 228us (473us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.786463:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 223 00002000:00100000:1.0:1788845018.786470:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1609728->1613823) req@ffff949f024d6d80 x1875739641393024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.786477:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d6d80 with x1875739641393024 ext(1609728->1613823) 00002000:00010000:1.0:1788845018.786480:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d6d80 00000100:00100000:0.0:1788845018.797876:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.805168:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641393152 00000100:00100000:2.0:1788845018.805205:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641393152 00002000:00100000:2.0:1788845018.805266:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1613824->1617919) req@ffff949f024d4e00 x1875739641393152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.805280:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d4e00 with x1875739641393152 ext(1613824->1617919) 00002000:00010000:2.0:1788845018.805286:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d4e00 00000100:00100000:2.0:1788845018.805434:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 224 00000100:00100000:2.0:1788845018.805446:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641393152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845018.805784:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641393152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 340us (7916us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845018.805793:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 224 00002000:00100000:2.0:1788845018.805800:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1613824->1617919) req@ffff949f024d4e00 x1875739641393152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.805808:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d4e00 with x1875739641393152 ext(1613824->1617919) 00002000:00010000:2.0:1788845018.805812:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d4e00 00000100:00100000:0.0:1788845018.809994:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.810106:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641393280 00000100:00100000:1.0:1788845018.810116:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641393280 00002000:00100000:1.0:1788845018.810170:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1617920->1622015) req@ffff949f024d4a80 x1875739641393280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.810181:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d4a80 with x1875739641393280 ext(1617920->1622015) 00002000:00010000:1.0:1788845018.810201:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d4a80 00000100:00100000:1.0:1788845018.810240:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 225 00000100:00100000:1.0:1788845018.810250:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641393280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845018.810494:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641393280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 244us (512us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845018.810502:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 225 00002000:00100000:1.0:1788845018.810508:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1617920->1622015) req@ffff949f024d4a80 x1875739641393280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845018.810517:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d4a80 with x1875739641393280 ext(1617920->1622015) 00002000:00010000:1.0:1788845018.810520:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d4a80 00000100:00100000:0.0:1788845018.816729:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.816826:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641393408 00000100:00100000:3.0:1788845018.816836:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641393408 00002000:00100000:3.0:1788845018.816916:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1622016->1626111) req@ffff949f024d4000 x1875739641393408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.816927:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d4000 with x1875739641393408 ext(1622016->1626111) 00002000:00010000:3.0:1788845018.816933:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d4000 00000100:00100000:3.0:1788845018.816963:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 226 00000100:00100000:3.0:1788845018.817031:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641393408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845018.817335:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641393408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 307us (615us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845018.817343:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 226 00002000:00100000:3.0:1788845018.817350:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1622016->1626111) req@ffff949f024d4000 x1875739641393408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.817357:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d4000 with x1875739641393408 ext(1622016->1626111) 00002000:00010000:3.0:1788845018.817362:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d4000 00000100:00100000:0.0:1788845018.826546:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.826640:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641393536 00000100:00100000:2.0:1788845018.826648:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641393536 00002000:00100000:2.0:1788845018.826685:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1626112->1630207) req@ffff949f024d6a00 x1875739641393536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.826693:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d6a00 with x1875739641393536 ext(1626112->1630207) 00002000:00010000:2.0:1788845018.826698:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d6a00 00000100:00100000:2.0:1788845018.826723:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 227 00000100:00100000:2.0:1788845018.826730:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641393536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845018.826925:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641393536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 195us (390us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845018.826943:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 227 00002000:00100000:2.0:1788845018.826948:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1626112->1630207) req@ffff949f024d6a00 x1875739641393536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 488/4536 e 0 to 0 dl 1788845029 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845018.826954:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d6a00 with x1875739641393536 ext(1626112->1630207) 00002000:00010000:2.0:1788845018.826956:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d6a00 00000100:00100000:0.0:1788845018.839653:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.839733:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641393664 00000100:00100000:2.0:1788845018.839744:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641393664 00000100:00100000:2.0:1788845018.839799:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 626 00000100:00100000:2.0:1788845018.839811:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641393664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845018.839831:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845018.839851:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abd40 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77d9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.840068:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.840084:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.840107:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845018.840134:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1777329 00000004:00200000:2.0:1788845018.840138:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845018.840157:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.840182:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abd40 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.840214:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.840223:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845018.840226:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845018.840229:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864000, rc 0) 00000100:00100000:2.0:1788845018.840365:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641393664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 556us (721us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845018.840372:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 626 00000100:00100000:0.0:1788845018.854835:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.854908:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.855180:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641393920 00000100:00100000:1.0:1788845018.855191:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641393920 00000100:00100000:1.0:1788845018.855314:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 398 00000100:00100000:1.0:1788845018.855326:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641393920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845018.855343:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845018.855361:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287abd4e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77e7 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.855423:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359adc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660175488:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845018.855519:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845018.855655:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641393792 00000100:00100000:2.0:1788845018.855660:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641393792 00000100:00100000:2.0:1788845018.855692:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 627 00000100:00100000:2.0:1788845018.855700:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641393792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845018.855710:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845018.855725:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abd55 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77e0 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.855756:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845018.855779:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660175616:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845018.855805:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008964780 going to sleep for 11 seconds 00002000:00010000:1.0:1788845018.859735:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845017 -> 1788845018 00000100:00100000:1.0:1788845018.859766:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359adc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660175488:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845018.859805:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287abd4e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77e7 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.859849:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287abd4e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77e7 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.859866:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba400, rc 0) 00000100:00100000:1.0:1788845018.859972:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641393920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4640us (5079us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845018.859981:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 398 00000004:00010000:2.0:1788845018.860187:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845016 -> 1788845018 00000100:00100000:2.0:1788845018.860206:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660175616:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845018.860220:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845018.860239:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abd55 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77e0 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.860252:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845018.860255:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845018.860260:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abd55 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77e0 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.860270:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845018.860303:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641393792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4592us (5466us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845018.860311:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 627 00000100:00100000:0.0:1788845018.868918:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.869075:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641394176 00000100:00100000:2.0:1788845018.869085:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641394176 00000100:00100000:2.0:1788845018.869371:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 399 00000100:00100000:2.0:1788845018.869384:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641394176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845018.869403:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845018.869424:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abd5c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77f5 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845018.869488:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660175744:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845018.869561:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008964780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845018.870392:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.870713:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641394048 00000100:00100000:2.0:1788845018.870720:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641394048 00000100:00100000:2.0:1788845018.870760:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 628 00000100:00100000:2.0:1788845018.870768:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641394048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845018.870784:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845018.870813:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd63 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77ee expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.870846:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845018.870875:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660175872:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845018.870939:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0348ed3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845018.873416:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660175744:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845018.873458:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abd5c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77f5 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.873484:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abd5c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77f5 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.873497:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864200, rc 0) 00000100:00100000:2.0:1788845018.873561:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641394176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4180us (4679us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845018.873570:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 399 00000100:00100000:2.0:1788845018.890601:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660175872:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845018.890633:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845018.890654:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd63 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77ee expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.890677:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845018.890680:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845018.890705:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd63 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77ee expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.890720:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864400, rc 0) 00000100:00100000:2.0:1788845018.890835:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641394048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 20068us (20445us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845018.890843:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 628 00000100:00100000:1.0:1788845018.907409:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.907512:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641394432 00000100:00100000:2.0:1788845018.907523:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641394432 00000100:00100000:2.0:1788845018.907581:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 629 00000100:00100000:2.0:1788845018.907591:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641394432:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845018.907644:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845018.907671:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abd39 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d2 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.907690:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845018.907695:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845018.907944:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.907960:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.907968:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.908001:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.908044:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 19 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845018.908059:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660176000:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845018.908137:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0348ed3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845018.915430:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.915507:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641394560 00000100:00100000:3.0:1788845018.915516:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641394560 00010000:00010000:3.0:1788845018.915555:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 20 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.915598:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 230 00000100:00100000:3.0:1788845018.915609:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641394560:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845018.915624:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845018.915631:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 20 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.915648:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 20 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.915671:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.915676:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.915686:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.915706:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.915726:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845018.915787:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641394560:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 180us (367us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845018.915794:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 230 00010000:00010000:2.0:1788845018.915912:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd47 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77d9 expref: 19 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845018.915928:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660176000:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845018.915947:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.915956:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.915961:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845018.916068:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967561 00000004:00010000:2.0:1788845018.916134:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788845018.916138:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845018 -> 1788845018 00000004:00010000:2.0:1788845018.916141:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845018 -> 1788845016 00000004:00010000:2.0:1788845018.916143:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845018 -> 1788845018 00000004:00010000:2.0:1788845018.916145:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:2.0:1788845018.916153:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000092c3ac2e reply state 000000005439d40c transno 4294967562 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845018.916173:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845018.916241:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641394432:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8651us (8842us total) trans 4294967562 rc 0/0 00000100:00100000:2.0:1788845018.916248:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 629 00000020:00080000:2.0:1788845018.916252:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845018.922490:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.922594:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641394688 00000100:00100000:1.0:1788845018.922603:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641394688 00002000:00010000:1.0:1788845018.922659:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1975993->18446744073709551615) 00002000:00100000:1.0:1788845018.922664:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d5c00 with x1875739641394688 ext(1975993->18446744073709551615) 00002000:00010000:1.0:1788845018.922678:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d5c00. 00000100:00100000:1.0:1788845018.922709:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 228 00000100:00100000:1.0:1788845018.922717:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641394688:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845018.922975:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1777329 -> 1975993 00002000:00010000:1.0:1788845018.922979:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845018 -> 1788845018 00002000:00010000:1.0:1788845018.922981:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845018 -> 1788845017 00002000:00010000:1.0:1788845018.922983:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845018 -> 1788845018 00002000:00010000:1.0:1788845018.922985:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 373 -> 373 00000100:00100000:1.0:1788845018.923036:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641394688:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 319us (555us total) trans 4294967371 rc 0/0 00000100:00100000:1.0:1788845018.923058:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 228 00002000:00010000:1.0:1788845018.923061:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1975993->18446744073709551615) 00002000:00100000:1.0:1788845018.923064:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d5c00 with x1875739641394688 ext(1975993->18446744073709551615) 00002000:00010000:1.0:1788845018.923069:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d5c00. 00010000:00010000:3.0:1788845018.923226:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.923236:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.923244:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.923250:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845018.923254:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.923262:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.923276:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.923279:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845018.923284:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.923290:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287abd6a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.935626:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.935725:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641394816 00000100:00100000:2.0:1788845018.935734:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641394816 00000100:00100000:2.0:1788845018.935794:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 630 00000100:00100000:2.0:1788845018.935805:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641394816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845018.935825:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845018.935845:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd71 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a77fc expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.935977:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abd78 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.935992:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abd78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.936003:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abd78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845018.936029:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1975993 00000004:00200000:2.0:1788845018.936033:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845018.936050:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abd78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.936076:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abd71 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77fc expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.936121:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abd78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77fc expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.936129:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845018.936132:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845018.936135:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845018.936182:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641394816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 379us (575us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845018.936190:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 630 00000100:00100000:0.0:1788845018.940096:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845018.940141:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.940218:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641395072 00000100:00100000:3.0:1788845018.940227:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641395072 00000100:00100000:3.0:1788845018.940287:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 400 00000100:00100000:3.0:1788845018.940308:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641395072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.940328:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.940346:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abd7f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a780a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.940406:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660176128:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845018.940492:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845018.940755:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641394944 00000100:00100000:1.0:1788845018.940760:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641394944 00000100:00100000:1.0:1788845018.940803:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 631 00000100:00100000:1.0:1788845018.940810:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641394944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845018.940821:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845018.940834:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287abd86 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7803 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.940862:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.940926:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660176256:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845018.940955:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788845018.946499:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845016 -> 1788845018 00000100:00100000:1.0:1788845018.946528:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660176256:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845018.946553:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845018.946570:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287abd86 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7803 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.946590:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845018.946593:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845018.946610:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287abd86 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7803 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.946627:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb400, rc 0) 00000100:00100000:1.0:1788845018.946686:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641394944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5875us (6607us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845018.946694:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 631 00002000:00010000:3.0:1788845018.946773:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845017 -> 1788845018 00000100:00100000:3.0:1788845018.946787:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660176128:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845018.946806:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abd7f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a780a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.946822:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abd7f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a780a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.946831:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845018.946850:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641395072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6545us (6709us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845018.946857:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 400 00000100:00100000:2.0:1788845018.950256:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.950300:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.950411:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641395328 00000100:00100000:3.0:1788845018.950420:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641395328 00000100:00100000:3.0:1788845018.950482:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 401 00000100:00100000:3.0:1788845018.950491:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641395328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.950509:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.950524:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287abd8d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7818 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.950577:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660176384:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845018.950649:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845018.950736:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641395200 00000100:00100000:0.0:1788845018.950741:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641395200 00000100:00100000:0.0:1788845018.950775:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 632 00000100:00100000:0.0:1788845018.950782:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641395200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845018.950791:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845018.950803:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abd94 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7811 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845018.950828:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.950848:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f024d7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660176512:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845018.950870:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845018.961541:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f024d7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660176512:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845018.961571:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845018.961589:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abd94 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7811 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.961610:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845018.961613:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845018.961618:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abd94 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7811 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.961633:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251be00, rc 0) 00000100:00100000:2.0:1788845018.961698:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641395200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10915us (11395us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845018.961707:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 632 00000100:00100000:3.0:1788845018.962341:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660176384:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845018.962366:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287abd8d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7818 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.962384:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287abd8d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7818 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.962395:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:3.0:1788845018.962433:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641395328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11943us (12187us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845018.962440:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 401 00000100:00100000:0.0:1788845018.970705:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845018.970781:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641395456 00000100:00100000:2.0:1788845018.970791:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641395456 00000100:00100000:2.0:1788845018.970862:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 333 00000100:00100000:2.0:1788845018.970871:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641395456:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845018.971168:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788845018.971172:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845018 -> 1788845018 00000004:00010000:2.0:1788845018.971175:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845018 -> 1788845016 00000004:00010000:2.0:1788845018.971177:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845018 -> 1788845018 00000004:00010000:2.0:1788845018.971180:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:2.0:1788845018.971239:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641395456:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 368us (542us total) trans 4294967563 rc 0/0 00000100:00100000:2.0:1788845018.971246:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 333 00000100:00100000:1.0:1788845018.976841:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.976957:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641395584 00000100:00100000:3.0:1788845018.976978:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641395584 00000100:00100000:3.0:1788845018.977035:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 633 00000100:00100000:3.0:1788845018.977044:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641395584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845018.977069:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845018.977099:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abd9b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a781f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.977197:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.977211:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.977222:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845018.977246:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1975993 00000004:00200000:3.0:1788845018.977250:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845018.977267:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.977538:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abd9b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.977558:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.977566:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.977569:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845018.977572:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:2.0:1788845018.977703:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:14891:x1875739641395584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 658us (869us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845018.977711:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 633 00000100:00100000:1.0:1788845018.983314:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.983520:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641395840 00000100:00100000:3.0:1788845018.983530:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641395840 00000100:00100000:3.0:1788845018.983597:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 634 00000100:00100000:3.0:1788845018.983607:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641395840:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845018.983661:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845018.983694:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abd78 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77fc expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.983713:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845018.983718:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845018.983971:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.983985:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.983991:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.984023:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.984040:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 20 pid: 11036 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.984052:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660176640:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845018.984140:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845018.987570:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660176640:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845018.987600:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.987673:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.987775:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641395968 00000100:00100000:1.0:1788845018.987783:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641395968 00010000:00010000:1.0:1788845018.987821:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 21 pid: 11036 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.987863:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 231 00000100:00100000:1.0:1788845018.987871:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2369:x1875739641395968:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845018.987921:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845018.987929:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 21 pid: 11036 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.987946:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 21 pid: 11036 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.987979:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287abda2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a781f expref: 21 pid: 11036 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.987995:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845018.988000:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.988016:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.988024:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.988046:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845018.988105:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641395968:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 233us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845018.988113:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 231 00010000:00010000:3.0:1788845018.988297:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.988315:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845018.988434:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967564 00000004:00010000:3.0:1788845018.988483:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845018.988487:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845018 -> 1788845019 00000004:00010000:3.0:1788845018.988490:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845016 -> 1788845016 00000004:00010000:3.0:1788845018.988492:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845018 -> 1788845019 00000004:00010000:3.0:1788845018.988494:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:3.0:1788845018.988500:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fd77bded reply state 000000003a7faedc transno 4294967565 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845018.988508:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845018.988571:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641395840:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4965us (5278us total) trans 4294967565 rc 0/0 00000100:00100000:3.0:1788845018.988577:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 634 00000020:00080000:3.0:1788845018.988581:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845018.990925:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.990941:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.990949:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.990955:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845018.990967:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.990975:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.991008:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845018.991011:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845018.991015:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845018.991022:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abda9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845018.991441:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845018.995799:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641396096 00000100:00100000:1.0:1788845018.995815:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641396096 00000100:00100000:1.0:1788845018.995915:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 635 00000100:00100000:1.0:1788845018.995926:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641396096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845018.995950:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845018.995983:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7826 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.996015:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.996025:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.996059:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.996076:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 19 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845018.996123:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660176768:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845018.996204:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845018.998379:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660176768:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845018.998421:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.998442:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845018.998445:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845018.998449:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845018.998457:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbac00, rc 0) 00000100:00100000:1.0:1788845018.998506:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641396096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2581us (7068us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845018.998515:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 635 00000100:00100000:0.0:1788845018.999010:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845018.999133:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641396224 00000100:00100000:3.0:1788845018.999140:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641396224 00002000:00100000:3.0:1788845018.999181:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02365f80 x1875739641396224/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 504/0 e 0 to 0 dl 1788845029 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845018.999191:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02365f80 with x1875739641396224 00010000:00010000:3.0:1788845018.999197:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 20 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845018.999214:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 20 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845018.999240:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f02365f80 00000100:00100000:3.0:1788845018.999271:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 16 00000100:00100000:3.0:1788845018.999280:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641396224:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845019.003621:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845019.003632:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845019 -> 1788845018 00000004:00010000:3.0:1788845019.003634:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845016 -> 1788845016 00000004:00010000:3.0:1788845019.003637:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845019 -> 1788845018 00000004:00010000:3.0:1788845019.003639:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845019.003708:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641396224:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4425us (4696us total) trans 4294967566 rc 0/0 00000100:00100000:3.0:1788845019.003716:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 16 00002000:00100000:3.0:1788845019.003727:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02365f80 x1875739641396224/t4294967566(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:124/0 lens 504/448 e 0 to 0 dl 1788845029 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.003733:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02365f80 with x1875739641396224 00010000:00010000:3.0:1788845019.003741:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 22 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.003756:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 22 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845019.003762:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f02365f80 00000100:00100000:0.0:1788845019.010599:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.010701:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641396352 00000100:00100000:3.0:1788845019.010709:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641396352 00010000:00010000:3.0:1788845019.010754:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 22 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.010795:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 232 00000100:00100000:3.0:1788845019.010805:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641396352:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845019.010820:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845019.010845:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 22 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.010864:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 22 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.010918:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abc83 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7746 expref: 22 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.010935:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845019.010942:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.010952:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.010972:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.011009:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.011033:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660176896:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845019.011113:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bf00 going to sleep for 11 seconds 00000100:00080000:0.0:1788845019.011643:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845019.018157:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a16680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660176896:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845019.018183:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845019.018239:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641396352:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7435us (7647us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845019.018250:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 232 00000100:00100000:2.0:1788845019.023042:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.023190:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641396480 00000100:00100000:0.0:1788845019.023200:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641396480 00000100:00100000:0.0:1788845019.023275:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 402 00000100:00100000:0.0:1788845019.023287:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641396480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.023321:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.023343:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a782d expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.023383:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.023392:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(10 6) 00010000:00010000:0.0:1788845019.023397:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 10) 00010000:00010000:0.0:1788845019.023402:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.023437:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.023455:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 8 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.023490:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660177024:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845019.023570:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00000100:00080000:3.0:1788845019.023692:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845019.034804:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.035117:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660177024:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845019.035159:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.035180:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251ba00, rc 0) 00000100:00100000:1.0:1788845019.035241:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641396480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11957us (12208us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845019.035250:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 402 00000100:00100000:3.0:1788845019.044419:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641396608 00000100:00100000:3.0:1788845019.044435:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641396608 00002000:00100000:3.0:1788845019.044506:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1404928->1437695) req@ffff949f02365c00 x1875739641396608/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.044517:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365c00 with x1875739641396608 ext(1404928->1437695) 00010000:00010000:3.0:1788845019.044527:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 9 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.044547:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 9 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845019.044554:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f02365c00 00000100:00100000:3.0:1788845019.044589:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 7 00000100:00100000:3.0:1788845019.044598:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641396608:12345-192.168.201.29@tcp:4:fsx.0 00000100:00080000:0.0:1788845019.045347:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845019.053402:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641396608:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 8796us (18599us total) trans 4294967372 rc 0/0 00000100:00100000:3.0:1788845019.053429:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 7 00002000:00100000:3.0:1788845019.053443:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1404928->1437695) req@ffff949f02365c00 x1875739641396608/t4294967372(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/448 e 0 to 0 dl 1788845030 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.053455:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365c00 with x1875739641396608 ext(1404928->1437695) 00010000:00010000:3.0:1788845019.053466:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 11 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.053490:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 11 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845019.053499:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f02365c00 00000100:00100000:2.0:1788845019.056627:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.066444:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641396736 00000100:00100000:1.0:1788845019.066460:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641396736 00010000:00010000:1.0:1788845019.066513:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 11 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.066565:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 233 00000100:00100000:1.0:1788845019.066576:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641396736:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845019.066593:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845019.066616:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 11 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.066636:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 11 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.066665:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abd2b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a77cb expref: 11 pid: 6682 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.066683:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 10) 00010000:00010000:1.0:1788845019.066688:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 10) 00010000:00010000:1.0:1788845019.066698:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.066707:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.066743:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.066772:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660177152:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845019.066878:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845019.072114:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660177152:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845019.072147:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845019.072201:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641396736:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5626us (15582us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845019.072211:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 233 00000100:00100000:3.0:1788845019.075112:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.076522:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641396864 00000100:00100000:2.0:1788845019.076534:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641396864 00002000:00010000:2.0:1788845019.076571:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1404972->18446744073709551615) 00002000:00100000:2.0:1788845019.076577:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a16680 with x1875739641396864 ext(1404972->18446744073709551615) 00002000:00010000:2.0:1788845019.076588:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a16680. 00000100:00100000:2.0:1788845019.076633:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 229 00000100:00100000:2.0:1788845019.076642:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641396864:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845019.077014:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1975993 -> 1404972 00002000:00010000:2.0:1788845019.077018:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845018 -> 1788845019 00002000:00010000:2.0:1788845019.077021:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845018 -> 1788845018 00002000:00010000:2.0:1788845019.077023:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845018 -> 1788845019 00002000:00010000:2.0:1788845019.077025:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 373 -> 373 00000100:00100000:2.0:1788845019.077097:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641396864:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 455us (1994us total) trans 4294967373 rc 0/0 00000100:00100000:2.0:1788845019.077105:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 229 00002000:00010000:2.0:1788845019.077109:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1404972->18446744073709551615) 00002000:00100000:2.0:1788845019.077126:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034a16680 with x1875739641396864 ext(1404972->18446744073709551615) 00002000:00010000:2.0:1788845019.077133:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034a16680. 00000100:00100000:2.0:1788845019.082616:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.082702:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641396992 00000100:00100000:0.0:1788845019.082712:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641396992 00000100:00100000:0.0:1788845019.082774:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 636 00000100:00100000:0.0:1788845019.082784:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:14891:x1875739641396992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.082810:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.082830:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abdbe lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7834 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.083039:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.083055:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.083066:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845019.083102:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1404972 00000004:00200000:0.0:1788845019.083106:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845019.083125:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.083150:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abdbe lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.083192:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.083202:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.083205:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845019.083209:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a000, rc 0) 00000100:00100000:0.0:1788845019.083262:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:14891:x1875739641396992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 480us (655us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845019.083269:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 636 00000100:00100000:3.0:1788845019.088511:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.088561:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.088642:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641397248 00000100:00100000:0.0:1788845019.088651:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641397248 00000100:00100000:0.0:1788845019.088701:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 403 00000100:00100000:0.0:1788845019.088710:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641397248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.088727:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.088743:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abdcc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7842 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.088797:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660177280:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845019.088879:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.089162:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641397120 00000100:00100000:2.0:1788845019.089166:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641397120 00000100:00100000:2.0:1788845019.089216:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 637 00000100:00100000:2.0:1788845019.089225:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641397120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.089236:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.089248:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abdd3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a783b expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.089276:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.089300:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660177408:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.089346:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0348ed3c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845019.101974:0:11036:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845018 -> 1788845019 00000004:00010000:2.0:1788845019.101985:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845016 -> 1788845018 00000004:00010000:2.0:1788845019.101987:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845018 -> 1788845019 00000100:00100000:2.0:1788845019.102013:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660177408:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.102037:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.102057:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abdd3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a783b expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.102078:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.102081:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.102114:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abdd3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a783b expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.102131:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845019.102182:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641397120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12958us (13681us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.102191:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 637 00000100:00100000:0.0:1788845019.103266:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660177280:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845019.103296:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abdcc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7842 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.103316:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000068851f6d/0xb2187301287abdcc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7842 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.103339:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251aa00, rc 0) 00000100:00100000:0.0:1788845019.103399:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641397248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 14689us (14837us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845019.103407:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 403 00000100:00100000:3.0:1788845019.116040:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.116148:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641397376 00000100:00100000:2.0:1788845019.116158:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641397376 00000100:00100000:2.0:1788845019.116222:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 638 00000100:00100000:2.0:1788845019.116232:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641397376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.116253:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.116274:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abdda lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7849 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.116326:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.116370:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660177536:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.116504:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0348ed3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845019.125819:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.125985:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641397504 00000100:00100000:2.0:1788845019.125996:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641397504 00000100:00100000:2.0:1788845019.126058:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 404 00000100:00100000:2.0:1788845019.126069:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641397504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.126097:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.126115:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abde1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7850 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.126199:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660177664:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.126271:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845019.139642:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660177664:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845019.139681:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abde1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7850 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.139706:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abde1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7850 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.139719:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865400, rc 0) 00000100:00100000:3.0:1788845019.139785:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641397504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13717us (13975us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845019.139806:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 404 00000100:00100000:2.0:1788845019.143807:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660177536:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.143838:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.143858:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abdda lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7849 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.143902:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.143907:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.143912:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abdda lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7849 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.143928:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845019.144131:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641397376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 27899us (28098us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.144140:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 638 00000100:00100000:1.0:1788845019.159028:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.159081:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.159177:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641397632 00000100:00100000:2.0:1788845019.159188:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641397632 00000100:00100000:2.0:1788845019.159255:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 639 00000100:00100000:2.0:1788845019.159267:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641397632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.159299:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.159319:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abde8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7857 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.159361:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.159416:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660177792:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.159499:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845019.159651:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641397760 00000100:00100000:3.0:1788845019.159655:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641397760 00000100:00100000:3.0:1788845019.159695:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 405 00000100:00100000:3.0:1788845019.159703:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641397760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.159712:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.159723:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdef lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a785e expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.159757:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660177920:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845019.159784:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845019.166241:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660177920:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845019.166281:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdef lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a785e expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.166318:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdef lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a785e expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.166332:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:3.0:1788845019.166394:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641397760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6691us (7312us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845019.166403:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 405 00000100:00100000:2.0:1788845019.166516:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660177792:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.166533:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.166545:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abde8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7857 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.166559:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.166562:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.166566:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287abde8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7857 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.166577:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845019.166601:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641397632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7338us (7583us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.166608:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 639 00000100:00100000:1.0:1788845019.177843:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.177979:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641397888 00000100:00100000:3.0:1788845019.177989:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641397888 00000100:00100000:3.0:1788845019.178121:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 406 00000100:00100000:3.0:1788845019.178133:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641397888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.178153:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.178174:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7865 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.178209:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.178218:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 10) 00010000:00010000:3.0:1788845019.178222:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 10) 00010000:00010000:3.0:1788845019.178227:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1183744->1187839] (req 1183744->1187839) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.178262:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.178280:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 6 pid: 6685 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.178303:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660178048:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845019.178388:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845019.181554:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.181761:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641398016 00000100:00100000:0.0:1788845019.181770:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641398016 00010000:00010000:0.0:1788845019.181810:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 7 pid: 6685 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.181854:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 234 00000100:00100000:0.0:1788845019.181864:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641398016:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845019.181878:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845019.181935:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 7 pid: 6685 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.181966:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 7 pid: 6685 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.181992:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 10) 00010000:00010000:0.0:1788845019.181996:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 10) 00010000:00010000:0.0:1788845019.182005:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.182015:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.182067:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.182092:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660178176:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845019.182164:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00010000:00010000:3.0:1788845019.184215:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c3cd486/0xb2187301287abdb7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a782d expref: 7 pid: 6685 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.184238:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660178048:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845019.184281:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.184305:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:3.0:1788845019.184365:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641397888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6234us (6531us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845019.184374:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 406 00000100:00100000:0.0:1788845019.193981:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660178176:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845019.194011:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845019.194069:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641398016:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 12205us (12525us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845019.194080:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 234 00000100:00100000:2.0:1788845019.194405:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.194496:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641398144 00000100:00100000:0.0:1788845019.194503:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641398144 00002000:00100000:0.0:1788845019.194546:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1183744->1187839) req@ffff94a0351f0700 x1875739641398144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.194557:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f0700 with x1875739641398144 ext(1183744->1187839) 00002000:00010000:0.0:1788845019.194561:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f0700 00000100:00100000:0.0:1788845019.194590:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 230 00000100:00100000:0.0:1788845019.194599:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.194802:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 204us (398us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.194810:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 230 00002000:00100000:0.0:1788845019.194817:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1183744->1187839) req@ffff94a0351f0700 x1875739641398144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.194844:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f0700 with x1875739641398144 ext(1183744->1187839) 00002000:00010000:0.0:1788845019.194847:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f0700 00000100:00100000:3.0:1788845019.215973:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.216090:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641398272 00000100:00100000:1.0:1788845019.216100:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641398272 00002000:00100000:1.0:1788845019.216149:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1187840->1191935) req@ffff94a03f6c4380 x1875739641398272/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845019.216160:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c4380 with x1875739641398272 ext(1187840->1191935) 00002000:00010000:1.0:1788845019.216165:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c4380 00000100:00100000:1.0:1788845019.216189:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 231 00000100:00100000:1.0:1788845019.216199:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845019.216454:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (490us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845019.216461:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 231 00002000:00100000:1.0:1788845019.216475:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1187840->1191935) req@ffff94a03f6c4380 x1875739641398272/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845019.216484:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c4380 with x1875739641398272 ext(1187840->1191935) 00002000:00010000:1.0:1788845019.216488:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c4380 00000100:00100000:1.0:1788845019.224064:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.224188:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641398400 00000100:00100000:2.0:1788845019.224199:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641398400 00002000:00100000:2.0:1788845019.224248:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1191936->1196031) req@ffff949f02362a00 x1875739641398400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.224260:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02362a00 with x1875739641398400 ext(1191936->1196031) 00002000:00010000:2.0:1788845019.224265:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02362a00 00000100:00100000:2.0:1788845019.224312:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 232 00000100:00100000:2.0:1788845019.224322:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.224565:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (509us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.224573:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 232 00002000:00100000:2.0:1788845019.224579:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1191936->1196031) req@ffff949f02362a00 x1875739641398400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.224587:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02362a00 with x1875739641398400 ext(1191936->1196031) 00002000:00010000:2.0:1788845019.224590:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02362a00 00000100:00100000:2.0:1788845019.240663:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.240769:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641398528 00000100:00100000:0.0:1788845019.240780:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641398528 00002000:00100000:0.0:1788845019.240834:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1196032->1200127) req@ffff94a0351f0e00 x1875739641398528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.240846:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f0e00 with x1875739641398528 ext(1196032->1200127) 00002000:00010000:0.0:1788845019.240852:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f0e00 00000100:00100000:0.0:1788845019.240927:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 233 00000100:00100000:0.0:1788845019.240938:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398528:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.241180:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398528:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (527us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.241188:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 233 00002000:00100000:0.0:1788845019.241195:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1196032->1200127) req@ffff94a0351f0e00 x1875739641398528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.241203:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f0e00 with x1875739641398528 ext(1196032->1200127) 00002000:00010000:0.0:1788845019.241206:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f0e00 00000100:00100000:0.0:1788845019.245597:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.245687:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641398656 00000100:00100000:3.0:1788845019.245697:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641398656 00002000:00100000:3.0:1788845019.245745:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1200128->1204223) req@ffff949f02366300 x1875739641398656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.245755:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02366300 with x1875739641398656 ext(1200128->1204223) 00002000:00010000:3.0:1788845019.245761:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02366300 00000100:00100000:3.0:1788845019.245792:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 234 00000100:00100000:3.0:1788845019.245802:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398656:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845019.246078:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398656:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 276us (489us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845019.246094:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 234 00002000:00100000:3.0:1788845019.246101:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1200128->1204223) req@ffff949f02366300 x1875739641398656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.246109:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02366300 with x1875739641398656 ext(1200128->1204223) 00002000:00010000:3.0:1788845019.246112:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02366300 00000100:00100000:1.0:1788845019.259205:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.259307:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641398784 00000100:00100000:2.0:1788845019.259317:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641398784 00002000:00100000:2.0:1788845019.259367:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1204224->1208319) req@ffff949f02361880 x1875739641398784/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.259378:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02361880 with x1875739641398784 ext(1204224->1208319) 00002000:00010000:2.0:1788845019.259383:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02361880 00000100:00100000:2.0:1788845019.259418:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 235 00000100:00100000:2.0:1788845019.259428:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398784:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.259655:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398784:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 228us (460us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.259663:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 235 00002000:00100000:2.0:1788845019.259670:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1204224->1208319) req@ffff949f02361880 x1875739641398784/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.259678:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02361880 with x1875739641398784 ext(1204224->1208319) 00002000:00010000:2.0:1788845019.259681:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02361880 00000100:00100000:2.0:1788845019.269726:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.270157:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641398912 00000100:00100000:0.0:1788845019.270167:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641398912 00002000:00100000:0.0:1788845019.270216:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1208320->1212415) req@ffff94a0351f1500 x1875739641398912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.270228:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f1500 with x1875739641398912 ext(1208320->1212415) 00002000:00010000:0.0:1788845019.270248:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f1500 00000100:00100000:0.0:1788845019.270285:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 236 00000100:00100000:0.0:1788845019.270295:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.270563:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641398912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 269us (847us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.270571:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 236 00002000:00100000:0.0:1788845019.270577:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1208320->1212415) req@ffff94a0351f1500 x1875739641398912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.270585:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f1500 with x1875739641398912 ext(1208320->1212415) 00002000:00010000:0.0:1788845019.270589:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f1500 00000100:00100000:0.0:1788845019.276379:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.276473:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641399040 00000100:00100000:3.0:1788845019.276498:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641399040 00002000:00100000:3.0:1788845019.276549:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1212416->1216511) req@ffff949f02365500 x1875739641399040/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.276560:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365500 with x1875739641399040 ext(1212416->1216511) 00002000:00010000:3.0:1788845019.276566:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365500 00000100:00100000:3.0:1788845019.276599:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 237 00000100:00100000:3.0:1788845019.276608:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641399040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845019.276856:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641399040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (487us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845019.276863:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 237 00002000:00100000:3.0:1788845019.276869:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1212416->1216511) req@ffff949f02365500 x1875739641399040/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.276877:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365500 with x1875739641399040 ext(1212416->1216511) 00002000:00010000:3.0:1788845019.276879:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365500 00000100:00100000:1.0:1788845019.280542:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.294183:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641399168 00000100:00100000:2.0:1788845019.294208:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641399168 00002000:00100000:2.0:1788845019.294274:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1220607) req@ffff949f02363100 x1875739641399168/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.294284:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02363100 with x1875739641399168 ext(1216512->1220607) 00002000:00010000:2.0:1788845019.294290:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02363100 00000100:00100000:2.0:1788845019.294333:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 238 00000100:00100000:2.0:1788845019.294344:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641399168:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.294584:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641399168:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 240us (14050us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.294591:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 238 00002000:00100000:2.0:1788845019.294598:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1220607) req@ffff949f02363100 x1875739641399168/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.294605:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02363100 with x1875739641399168 ext(1216512->1220607) 00002000:00010000:2.0:1788845019.294608:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02363100 00000100:00100000:2.0:1788845019.308433:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.308547:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641399296 00000100:00100000:0.0:1788845019.308556:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641399296 00002000:00100000:0.0:1788845019.308605:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1220608->1224703) req@ffff94a019d90e00 x1875739641399296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.308615:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d90e00 with x1875739641399296 ext(1220608->1224703) 00002000:00010000:0.0:1788845019.308621:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d90e00 00000100:00100000:0.0:1788845019.308655:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 239 00000100:00100000:0.0:1788845019.308666:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641399296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.308936:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641399296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 271us (512us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.308944:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 239 00002000:00100000:0.0:1788845019.308951:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1220608->1224703) req@ffff94a019d90e00 x1875739641399296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.308979:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d90e00 with x1875739641399296 ext(1220608->1224703) 00002000:00010000:0.0:1788845019.308982:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d90e00 00000100:00100000:3.0:1788845019.317811:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.318867:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641399424 00000100:00100000:2.0:1788845019.318880:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641399424 00002000:00100000:2.0:1788845019.318965:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1224704->1228799) req@ffff949f02a48a80 x1875739641399424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.318978:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a48a80 with x1875739641399424 ext(1224704->1228799) 00002000:00010000:2.0:1788845019.318983:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a48a80 00000100:00100000:2.0:1788845019.319008:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 240 00000100:00100000:2.0:1788845019.319027:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641399424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.319460:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641399424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 433us (1657us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.319468:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 240 00002000:00100000:2.0:1788845019.319474:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1224704->1228799) req@ffff949f02a48a80 x1875739641399424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.319482:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a48a80 with x1875739641399424 ext(1224704->1228799) 00002000:00010000:2.0:1788845019.319485:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a48a80 00000100:00100000:1.0:1788845019.326706:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.328340:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641399552 00000100:00100000:3.0:1788845019.328353:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641399552 00002000:00100000:3.0:1788845019.328409:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1228800->1232895) req@ffff949f02361f80 x1875739641399552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.328422:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02361f80 with x1875739641399552 ext(1228800->1232895) 00002000:00010000:3.0:1788845019.328428:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02361f80 00000100:00100000:3.0:1788845019.328466:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 241 00000100:00100000:3.0:1788845019.328476:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641399552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845019.329346:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641399552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 868us (2646us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845019.329370:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 241 00002000:00100000:3.0:1788845019.329380:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1228800->1232895) req@ffff949f02361f80 x1875739641399552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.329388:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02361f80 with x1875739641399552 ext(1228800->1232895) 00002000:00010000:3.0:1788845019.329393:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02361f80 00000100:00100000:3.0:1788845019.340496:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.340905:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641399680 00000100:00100000:0.0:1788845019.340915:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641399680 00002000:00100000:0.0:1788845019.340963:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1232896->1236991) req@ffff949f02a48000 x1875739641399680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.340972:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a48000 with x1875739641399680 ext(1232896->1236991) 00002000:00010000:0.0:1788845019.340976:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a48000 00000100:00100000:0.0:1788845019.341001:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 242 00000100:00100000:0.0:1788845019.341008:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641399680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.341447:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641399680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 437us (960us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.341453:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 242 00002000:00100000:0.0:1788845019.341458:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1232896->1236991) req@ffff949f02a48000 x1875739641399680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.341464:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a48000 with x1875739641399680 ext(1232896->1236991) 00002000:00010000:0.0:1788845019.341466:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a48000 00000100:00100000:0.0:1788845019.346635:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.346793:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641399808 00000100:00100000:1.0:1788845019.346801:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641399808 00002000:00100000:1.0:1788845019.346841:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1236992->1241087) req@ffff949f02365180 x1875739641399808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845019.346849:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365180 with x1875739641399808 ext(1236992->1241087) 00002000:00010000:1.0:1788845019.346854:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365180 00000100:00100000:1.0:1788845019.346895:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 243 00000100:00100000:1.0:1788845019.346902:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641399808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845019.347114:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641399808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 211us (487us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845019.347122:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 243 00002000:00100000:1.0:1788845019.347129:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1236992->1241087) req@ffff949f02365180 x1875739641399808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845019.347137:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365180 with x1875739641399808 ext(1236992->1241087) 00002000:00010000:1.0:1788845019.347141:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365180 00000100:00100000:1.0:1788845019.353409:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.353511:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641399936 00000100:00100000:0.0:1788845019.353521:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641399936 00002000:00100000:0.0:1788845019.353569:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1241088->1245183) req@ffff949f02360380 x1875739641399936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.353581:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360380 with x1875739641399936 ext(1241088->1245183) 00002000:00010000:0.0:1788845019.353602:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360380 00000100:00100000:0.0:1788845019.353634:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 244 00000100:00100000:0.0:1788845019.353644:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641399936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.353932:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641399936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 288us (531us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.353940:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 244 00002000:00100000:0.0:1788845019.353947:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1241088->1245183) req@ffff949f02360380 x1875739641399936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.353955:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360380 with x1875739641399936 ext(1241088->1245183) 00002000:00010000:0.0:1788845019.353959:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360380 00000100:00100000:0.0:1788845019.359162:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.359281:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641400064 00000100:00100000:3.0:1788845019.359291:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641400064 00002000:00100000:3.0:1788845019.359341:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff949f02367480 x1875739641400064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.359351:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02367480 with x1875739641400064 ext(1245184->1249279) 00002000:00010000:3.0:1788845019.359357:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02367480 00000100:00100000:3.0:1788845019.359460:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 245 00000100:00100000:3.0:1788845019.359471:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641400064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845019.359768:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641400064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 297us (611us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845019.359775:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 245 00002000:00100000:3.0:1788845019.359782:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff949f02367480 x1875739641400064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.359790:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02367480 with x1875739641400064 ext(1245184->1249279) 00002000:00010000:3.0:1788845019.359793:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02367480 00000100:00100000:3.0:1788845019.364977:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.365096:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641400192 00000100:00100000:1.0:1788845019.365119:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641400192 00002000:00100000:1.0:1788845019.365168:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1253375) req@ffff949f02a4aa00 x1875739641400192/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845019.365189:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a4aa00 with x1875739641400192 ext(1249280->1253375) 00002000:00010000:1.0:1788845019.365196:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a4aa00 00000100:00100000:1.0:1788845019.365223:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 246 00000100:00100000:1.0:1788845019.365233:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641400192:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845019.365457:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641400192:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 224us (488us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845019.365465:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 246 00002000:00100000:1.0:1788845019.365471:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1253375) req@ffff949f02a4aa00 x1875739641400192/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845019.365479:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a4aa00 with x1875739641400192 ext(1249280->1253375) 00002000:00010000:1.0:1788845019.365483:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a4aa00 00000100:00100000:1.0:1788845019.370354:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.370634:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641400320 00000100:00100000:3.0:1788845019.370659:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641400320 00002000:00100000:3.0:1788845019.370708:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1183744->1253375) req@ffff949f02363800 x1875739641400320/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.370720:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02363800 with x1875739641400320 ext(1183744->1253375) 00002000:00010000:3.0:1788845019.370726:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02363800 00000100:00100000:3.0:1788845019.370763:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 247 00000100:00100000:3.0:1788845019.370773:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641400320:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845019.375316:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641400320:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4533us (4962us total) trans 4294967374 rc 0/0 00000100:00100000:3.0:1788845019.375332:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 247 00002000:00100000:3.0:1788845019.375347:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1183744->1253375) req@ffff949f02363800 x1875739641400320/t4294967374(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/448 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.375358:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02363800 with x1875739641400320 ext(1183744->1253375) 00002000:00010000:3.0:1788845019.375364:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02363800 00000100:00100000:0.0:1788845019.379234:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.379384:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641400448 00000100:00100000:2.0:1788845019.379394:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641400448 00000100:00100000:2.0:1788845019.379447:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 407 00000100:00100000:2.0:1788845019.379459:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641400448:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:2.0:1788845019.413105:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641400448:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 33636us (33870us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845019.413118:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 407 00000001:00080000:3.0:1788845019.415936:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967369 is committed 00000001:00080000:3.0:1788845019.415963:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967371 is committed 00000001:00080000:3.0:1788845019.415973:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967373 is committed 00000001:00080000:3.0:1788845019.415979:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967374 is committed 00000100:00100000:0.0:1788845019.416140:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.416288:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641400576 00000100:00100000:2.0:1788845019.416298:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641400576 00000100:00100000:2.0:1788845019.416358:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 640 00000100:00100000:2.0:1788845019.416369:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641400576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.416390:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.416411:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287abdfd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a786c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.416450:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.416510:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660178304:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.416591:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.419049:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660178304:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.419074:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.419108:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287abdfd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a786c expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.419129:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.419132:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.419138:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287abdfd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a786c expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.419154:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865e00, rc 0) 00000100:00100000:2.0:1788845019.419224:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641400576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2857us (3088us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.419233:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 640 00000100:00100000:0.0:1788845019.421924:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.422507:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641400704 00000100:00100000:2.0:1788845019.422518:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641400704 00000100:00100000:2.0:1788845019.422579:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 641 00000100:00100000:2.0:1788845019.422589:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641400704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.422621:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.422642:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287abe04 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7873 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.422680:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.422730:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891b100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660178432:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.422808:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.425104:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891b100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660178432:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.425132:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.425149:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287abe04 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7873 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.425169:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.425173:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.425177:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287abe04 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7873 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.425265:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865c00, rc 0) 00000100:00100000:2.0:1788845019.425351:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641400704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2762us (3466us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.425359:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 641 00000100:00100000:2.0:1788845019.428370:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.428592:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641400832 00000100:00100000:3.0:1788845019.428611:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641400832 00000100:00100000:3.0:1788845019.428684:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 642 00000100:00100000:3.0:1788845019.428695:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641400832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.428713:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.428733:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abe0b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a787a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.428837:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287abe12 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.428853:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287abe12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.428865:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287abe12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845019.428928:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1404972 00000004:00200000:3.0:1788845019.428933:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845019.428950:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287abe12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.428986:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abe0b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a787a expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.429004:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287abe12 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a787a expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.429013:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845019.429016:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845019.429020:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:3.0:1788845019.429072:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641400832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 379us (711us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845019.429091:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 642 00000100:00080000:0.0:1788845019.430510:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845019.433484:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.433555:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641400960 00000100:00100000:3.0:1788845019.433566:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641400960 00000100:00100000:3.0:1788845019.433646:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 408 00000100:00100000:3.0:1788845019.433656:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641400960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.433674:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.433691:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abe19 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7881 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.433749:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660178560:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845019.433829:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d20900 going to sleep for 11 seconds 00002000:00010000:3.0:1788845019.436728:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845018 -> 1788845019 00000100:00100000:3.0:1788845019.436761:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660178560:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845019.436797:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abe19 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7881 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.436822:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abe19 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7881 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.436833:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845019.436930:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641400960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3275us (3454us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845019.436939:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 408 00000100:00100000:2.0:1788845019.440399:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.440566:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641401088 00000100:00100000:3.0:1788845019.440576:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641401088 00000100:00100000:3.0:1788845019.440640:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 409 00000100:00100000:3.0:1788845019.440650:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641401088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.440668:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.440697:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287abe20 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7888 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.440751:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660178688:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845019.440833:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d20900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845019.443556:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660178688:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845019.443606:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287abe20 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7888 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.443633:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287abe20 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7888 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.443645:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:3.0:1788845019.443704:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641401088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3055us (3316us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845019.443714:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 409 00000100:00100000:1.0:1788845019.449657:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.449759:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641401216 00000100:00100000:0.0:1788845019.449768:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641401216 00000100:00100000:0.0:1788845019.449835:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 643 00000100:00100000:0.0:1788845019.449845:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641401216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.449867:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.449927:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a800/0xb2187301287abe27 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a788f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.449975:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.450032:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660178816:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845019.450132:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000004:00010000:0.0:1788845019.452497:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845018 -> 1788845019 00000100:00100000:0.0:1788845019.452529:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660178816:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845019.452557:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845019.452577:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a800/0xb2187301287abe27 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a788f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.452597:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.452600:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845019.452605:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a800/0xb2187301287abe27 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a788f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.452620:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a800, rc 0) 00000100:00100000:0.0:1788845019.452677:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641401216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2833us (3030us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845019.452686:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 643 00000100:00100000:3.0:1788845019.456042:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.456306:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641401344 00000100:00100000:2.0:1788845019.456316:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641401344 00000100:00100000:2.0:1788845019.456375:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 644 00000100:00100000:2.0:1788845019.456384:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2367:x1875739641401344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.456406:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.456427:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abe2e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7896 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.456482:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.456523:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660178944:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.456609:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.459807:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660178944:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.459835:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.459854:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abe2e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7896 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.459875:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.459879:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.460031:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abe2e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7896 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.460048:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845019.460148:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2367:x1875739641401344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3764us (4117us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.460156:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 644 00000100:00100000:3.0:1788845019.463487:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.463594:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641401472 00000100:00100000:1.0:1788845019.463674:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641401472 00000100:00100000:1.0:1788845019.463766:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 410 00000100:00100000:1.0:1788845019.463778:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641401472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845019.463797:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845019.463819:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287abe35 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a789d expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.463911:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02362300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660179072:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845019.464019:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845019.466789:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02362300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660179072:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845019.466834:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287abe35 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a789d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.466861:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287abe35 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a789d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.466926:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb600, rc 0) 00000100:00100000:3.0:1788845019.467102:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641401472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3325us (3622us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845019.467112:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 410 00000100:00100000:0.0:1788845019.470694:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.470825:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641401600 00000100:00100000:2.0:1788845019.470834:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641401600 00002000:00100000:2.0:1788845019.470912:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366d80 x1875739641401600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.470924:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366d80 with x1875739641401600 00000100:00100000:2.0:1788845019.470961:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 334 00000100:00100000:2.0:1788845019.470985:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641401600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.471452:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641401600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 466us (765us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.471461:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 334 00002000:00100000:2.0:1788845019.471469:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366d80 x1875739641401600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.471477:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366d80 with x1875739641401600 00000100:00100000:2.0:1788845019.475474:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.475579:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641401728 00000100:00100000:0.0:1788845019.475589:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641401728 00002000:00100000:0.0:1788845019.475705:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fad80 x1875739641401728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.475717:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fad80 with x1875739641401728 00000100:00100000:0.0:1788845019.475751:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 335 00000100:00100000:0.0:1788845019.475760:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641401728:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.476522:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641401728:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 759us (1056us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.476531:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 335 00002000:00100000:0.0:1788845019.476539:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fad80 x1875739641401728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.476548:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fad80 with x1875739641401728 00000100:00100000:3.0:1788845019.480148:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.480311:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641401856 00000100:00100000:2.0:1788845019.480321:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641401856 00002000:00100000:2.0:1788845019.480370:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02a49f80 x1875739641401856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.480381:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02a49f80 with x1875739641401856 00000100:00100000:2.0:1788845019.480462:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 336 00000100:00100000:2.0:1788845019.480474:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641401856:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.480742:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641401856:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 271us (605us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.480750:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 336 00002000:00100000:2.0:1788845019.480757:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02a49f80 x1875739641401856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.480765:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02a49f80 with x1875739641401856 00000100:00100000:0.0:1788845019.484734:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.484843:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641401984 00000100:00100000:3.0:1788845019.484854:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641401984 00002000:00100000:3.0:1788845019.484919:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364e00 x1875739641401984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.484931:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364e00 with x1875739641401984 00000100:00100000:3.0:1788845019.485054:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 337 00000100:00100000:3.0:1788845019.485064:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641401984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845019.485422:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641401984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 359us (700us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845019.485430:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 337 00002000:00100000:3.0:1788845019.485437:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364e00 x1875739641401984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.485445:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364e00 with x1875739641401984 00000100:00100000:2.0:1788845019.489153:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.489312:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641402112 00000100:00100000:0.0:1788845019.489323:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641402112 00002000:00100000:0.0:1788845019.489375:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9180 x1875739641402112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.489386:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9180 with x1875739641402112 00000100:00100000:0.0:1788845019.489453:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 338 00000100:00100000:0.0:1788845019.489463:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641402112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.489775:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641402112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 311us (631us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.489782:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 338 00002000:00100000:0.0:1788845019.489789:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9180 x1875739641402112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.489797:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9180 with x1875739641402112 00000100:00100000:2.0:1788845019.493578:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.493712:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641402240 00000100:00100000:0.0:1788845019.493721:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641402240 00002000:00100000:0.0:1788845019.493768:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9c00 x1875739641402240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.493799:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9c00 with x1875739641402240 00000100:00100000:0.0:1788845019.493834:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 339 00000100:00100000:0.0:1788845019.493843:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641402240:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.494302:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641402240:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 458us (730us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.494311:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 339 00002000:00100000:0.0:1788845019.494318:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9c00 x1875739641402240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.494325:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9c00 with x1875739641402240 00000100:00100000:1.0:1788845019.498248:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.498424:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641402368 00000100:00100000:2.0:1788845019.498448:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641402368 00002000:00100000:2.0:1788845019.498498:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02361500 x1875739641402368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.498510:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02361500 with x1875739641402368 00000100:00100000:2.0:1788845019.498554:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 340 00000100:00100000:2.0:1788845019.498565:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641402368:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.498824:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641402368:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (585us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.498832:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 340 00002000:00100000:2.0:1788845019.498838:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02361500 x1875739641402368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.498846:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02361500 with x1875739641402368 00000100:00100000:2.0:1788845019.502342:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.502429:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641402496 00000100:00100000:3.0:1788845019.502439:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641402496 00002000:00100000:3.0:1788845019.502486:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a039cd8450 x1875739641402496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.502497:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a039cd8450 with x1875739641402496 00000100:00100000:3.0:1788845019.502551:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 341 00000100:00100000:3.0:1788845019.502561:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a039cd8450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641402496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845019.502835:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a039cd8450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641402496:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (502us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845019.502842:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 341 00002000:00100000:3.0:1788845019.502849:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a039cd8450 x1875739641402496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845019.502857:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a039cd8450 with x1875739641402496 00000100:00100000:3.0:1788845019.508286:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.508460:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641402624 00000100:00100000:2.0:1788845019.508471:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641402624 00002000:00100000:2.0:1788845019.508526:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02a49f80 x1875739641402624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.508538:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02a49f80 with x1875739641402624 00000100:00100000:2.0:1788845019.508576:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 342 00000100:00100000:2.0:1788845019.508586:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641402624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.508857:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641402624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 272us (582us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.508866:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 342 00002000:00100000:2.0:1788845019.508873:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02a49f80 x1875739641402624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.509107:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02a49f80 with x1875739641402624 00000100:00100000:1.0:1788845019.512291:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.512461:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641402752 00000100:00100000:0.0:1788845019.512471:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641402752 00002000:00100000:0.0:1788845019.512523:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02361180 x1875739641402752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.512535:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02361180 with x1875739641402752 00000100:00100000:0.0:1788845019.512575:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 343 00000100:00100000:0.0:1788845019.512585:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641402752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.512837:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641402752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (554us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.512845:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 343 00002000:00100000:0.0:1788845019.512853:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02361180 x1875739641402752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.512860:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02361180 with x1875739641402752 00000100:00100000:3.0:1788845019.516797:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.516914:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641402880 00000100:00100000:2.0:1788845019.516925:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641402880 00002000:00100000:2.0:1788845019.516984:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02a48380 x1875739641402880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.516996:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02a48380 with x1875739641402880 00000100:00100000:2.0:1788845019.517037:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 344 00000100:00100000:2.0:1788845019.517048:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641402880:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.517516:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641402880:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 468us (729us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.517524:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 344 00002000:00100000:2.0:1788845019.517531:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02a48380 x1875739641402880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.517539:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02a48380 with x1875739641402880 00000100:00100000:3.0:1788845019.521156:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.521417:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641403008 00000100:00100000:0.0:1788845019.521441:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641403008 00002000:00100000:0.0:1788845019.521491:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02a4b800 x1875739641403008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.521501:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02a4b800 with x1875739641403008 00000100:00100000:0.0:1788845019.521536:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 345 00000100:00100000:0.0:1788845019.521545:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641403008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.521787:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641403008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 242us (640us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.521795:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 345 00002000:00100000:0.0:1788845019.521801:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02a4b800 x1875739641403008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.521808:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02a4b800 with x1875739641403008 00000100:00100000:1.0:1788845019.530776:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.530860:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641403136 00000100:00100000:0.0:1788845019.530870:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641403136 00002000:00100000:0.0:1788845019.530944:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02363480 x1875739641403136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.530964:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02363480 with x1875739641403136 00000100:00100000:0.0:1788845019.530996:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 346 00000100:00100000:0.0:1788845019.531006:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641403136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.531427:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641403136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 421us (658us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.531436:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 346 00002000:00100000:0.0:1788845019.531443:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02363480 x1875739641403136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.531471:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02363480 with x1875739641403136 00000100:00100000:3.0:1788845019.536295:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.536484:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641403264 00000100:00100000:2.0:1788845019.536495:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641403264 00000100:00100000:2.0:1788845019.536561:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 645 00000100:00100000:2.0:1788845019.536570:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641403264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.536591:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.536620:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abe3c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78a4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.536656:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.536710:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660179200:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.536776:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.539406:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660179200:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.539438:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.539460:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abe3c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78a4 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.539482:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.539486:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.539491:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abe3c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78a4 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.539505:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845019.539563:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641403264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2994us (3279us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.539571:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 645 00000100:00100000:1.0:1788845019.544351:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.544451:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641403392 00000100:00100000:0.0:1788845019.544462:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641403392 00000100:00100000:0.0:1788845019.544523:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 646 00000100:00100000:0.0:1788845019.544533:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2369:x1875739641403392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.544554:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.544574:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b200/0xb2187301287abe43 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78ab expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.544622:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.544666:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660179328:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845019.544747:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845019.548423:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660179328:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845019.548453:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845019.548472:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b200/0xb2187301287abe43 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ab expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.548494:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.548497:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845019.548502:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b200/0xb2187301287abe43 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ab expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.548516:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b200, rc 0) 00000100:00100000:0.0:1788845019.548577:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2369:x1875739641403392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4046us (4236us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845019.548669:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 646 00000100:00100000:3.0:1788845019.552501:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.552589:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641403520 00000100:00100000:0.0:1788845019.552607:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641403520 00000100:00100000:0.0:1788845019.552671:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 647 00000100:00100000:0.0:1788845019.552681:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641403520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.552701:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.552723:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe4a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78b2 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.552758:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.552793:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660179456:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845019.552877:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845019.555635:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660179456:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845019.555664:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845019.555683:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe4a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78b2 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.555704:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.555707:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845019.555712:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78b2 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.555727:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251ae00, rc 0) 00000100:00100000:0.0:1788845019.555787:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641403520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3107us (3295us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845019.555812:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 647 00000100:00100000:3.0:1788845019.559329:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.559477:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641403648 00000100:00100000:2.0:1788845019.559487:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641403648 00000100:00100000:2.0:1788845019.559540:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 648 00000100:00100000:2.0:1788845019.559551:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641403648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.559571:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.559591:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abe51 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78b9 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.559645:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.559702:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660179584:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.559787:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.562529:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660179584:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.562556:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.562575:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abe51 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78b9 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.562594:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.562608:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.562613:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abe51 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78b9 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.562627:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845019.562685:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641403648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3136us (3364us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.562693:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 648 00000100:00100000:1.0:1788845019.567492:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.567576:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641403776 00000100:00100000:2.0:1788845019.567586:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641403776 00000100:00100000:2.0:1788845019.567663:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 649 00000100:00100000:2.0:1788845019.567674:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:14891:x1875739641403776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.567697:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.567719:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78c0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.567749:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.567773:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c0 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.567811:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.567831:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 22 pid: 5839 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.567846:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660179712:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845019.568447:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.571098:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660179712:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845019.571140:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c0 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.571159:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.571162:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.571167:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c0 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.571188:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00000100:00100000:2.0:1788845019.571249:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641403776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3577us (3766us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845019.571258:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 649 00000100:00100000:3.0:1788845019.573402:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.573736:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641403904 00000100:00100000:2.0:1788845019.573759:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641403904 00010000:00010000:2.0:1788845019.573795:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 23 pid: 5839 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.573836:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 235 00000100:00100000:2.0:1788845019.573845:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a49180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641403904:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845019.573860:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845019.573922:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 23 pid: 5839 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.573938:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 23 pid: 5839 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.574133:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287abdb0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7826 expref: 23 pid: 5839 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.574153:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.574160:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c0 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.574172:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c0 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.574190:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c0 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.574229:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c0 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.574362:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660179840:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845019.574450:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.578418:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660179840:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845019.578447:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845019.578507:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a49180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:2369:x1875739641403904:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4663us (5115us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845019.578520:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 235 00000100:00100000:3.0:1788845019.592641:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.594101:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641404160 00000100:00100000:2.0:1788845019.594115:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641404160 00002000:00100000:2.0:1788845019.594189:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1409023) req@ffff949f02a4b480 x1875739641404160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 504/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.594203:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a4b480 with x1875739641404160 ext(1048576->1409023) 00002000:00010000:2.0:1788845019.594209:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a4b480 00000100:00100000:2.0:1788845019.594247:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 248 00000100:00100000:2.0:1788845019.594259:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641404160:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845019.601230:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.602635:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641404032 00000100:00100000:0.0:1788845019.602648:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641404032 00002000:00100000:0.0:1788845019.602704:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02362300 x1875739641404032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.602717:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02362300 with x1875739641404032 00000100:00100000:0.0:1788845019.602761:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 347 00000100:00100000:0.0:1788845019.602771:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641404032:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.632283:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641404160:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 38013us (39640us total) trans 0 rc 290816/290816 00000100:00100000:2.0:1788845019.632298:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 248 00002000:00100000:2.0:1788845019.632310:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1409023) req@ffff949f02a4b480 x1875739641404160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 504/440 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 290816/290816 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.632321:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a4b480 with x1875739641404160 ext(1048576->1409023) 00002000:00010000:2.0:1788845019.632328:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a4b480 00000100:00100000:0.0:1788845019.660787:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641404032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 58005us (59555us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1788845019.660802:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 347 00002000:00100000:0.0:1788845019.660816:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02362300 x1875739641404032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/440 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.660827:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02362300 with x1875739641404032 00000100:00100000:1.0:1788845019.668614:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.668728:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.668808:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641404416 00000100:00100000:2.0:1788845019.668819:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641404416 00000100:00100000:2.0:1788845019.668907:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 411 00000100:00100000:2.0:1788845019.668918:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641404416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.668937:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.668965:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287abe5f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78ce expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.669024:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660179968:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.669125:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845019.671391:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641404288 00000100:00100000:0.0:1788845019.671406:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641404288 00000100:00100000:0.0:1788845019.671479:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 650 00000100:00100000:0.0:1788845019.671489:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+22:2368:x1875739641404288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.671508:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.671529:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78c7 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.671563:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845019.671593:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845019.671602:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.671617:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.671620:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845019.671624:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251ae00, rc 0) 00000100:00100000:0.0:1788845019.671766:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641404288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 279us (3163us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845019.671772:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 650 00000100:00100000:2.0:1788845019.674257:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660179968:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845019.674298:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287abe5f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ce expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.674322:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287abe5f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ce expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.674335:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845019.674389:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641404416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5472us (5661us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.674398:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 411 00000100:00100000:2.0:1788845019.681138:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.681375:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641404544 00000100:00100000:3.0:1788845019.681386:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641404544 00000100:00100000:3.0:1788845019.681457:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 412 00000100:00100000:3.0:1788845019.681468:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641404544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.681487:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.681504:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287abe6d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78d5 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.681564:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660180096:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845019.682327:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d20540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.696224:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660180096:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845019.696270:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287abe6d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78d5 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.696315:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287abe6d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78d5 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.696328:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:2.0:1788845019.696386:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641404544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 14919us (15257us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.696395:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 412 00000100:00100000:0.0:1788845019.702665:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.702758:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641404672 00000100:00100000:3.0:1788845019.702769:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641404672 00000100:00100000:3.0:1788845019.702836:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 413 00000100:00100000:3.0:1788845019.702847:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641404672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.702869:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.702924:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287abe74 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78dc expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.703003:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a4bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660180224:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845019.703107:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d20540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845019.709777:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a4bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660180224:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845019.709839:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287abe74 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78dc expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.709879:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287abe74 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78dc expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.709931:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:0.0:1788845019.710441:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641404672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7593us (7785us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845019.710453:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 413 00000100:00100000:1.0:1788845019.716495:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.716727:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641404800 00000100:00100000:3.0:1788845019.716742:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641404800 00000100:00100000:3.0:1788845019.716822:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 414 00000100:00100000:3.0:1788845019.716836:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641404800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.716864:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.717209:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abe7b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78e3 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.717392:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660180352:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.717513:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.721221:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660180352:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845019.721263:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abe7b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78e3 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.721289:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287abe7b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78e3 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.721303:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:2.0:1788845019.721381:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641404800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4549us (5133us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.721389:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 414 00000100:00100000:3.0:1788845019.729799:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:1.0:1788845019.730422:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845019.731121:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641404928 00000100:00100000:0.0:1788845019.731137:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641404928 00000100:00100000:0.0:1788845019.731316:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 651 00000100:00100000:0.0:1788845019.731342:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641404928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.731365:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.731368:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788845019.731411:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abe58 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c0 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.731432:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.731437:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845019.731451:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78ea expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.731475:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.731485:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.731517:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.731535:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 22 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.731548:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660180480:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845019.731639:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845019.737164:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.738219:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641405056 00000100:00100000:3.0:1788845019.738236:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641405056 00010000:00010000:3.0:1788845019.738528:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 23 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.738604:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 236 00000100:00100000:3.0:1788845019.738614:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008917050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641405056:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845019.738631:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845019.738686:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 23 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.738708:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 23 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.738737:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845019.738743:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.738758:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.738768:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.738820:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845019.738838:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a4ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660180608:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845019.739595:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d20f00 going to sleep for 11 seconds 00010000:00010000:0.0:1788845019.741241:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe66 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78c7 expref: 23 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.741290:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660180480:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845019.741321:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.741339:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.741342:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845019.741346:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b800, rc 0) 00000100:00100000:0.0:1788845019.741417:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641404928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10077us (11633us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845019.741426:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 651 00000100:00100000:3.0:1788845019.751296:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a4ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660180608:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845019.751319:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.751421:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641405184 00000100:00100000:2.0:1788845019.751433:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641405184 00010000:00010000:3.0:1788845019.751458:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845019.751521:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 348 00000100:00100000:2.0:1788845019.751548:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641405184:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845019.751811:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788845019.751815:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845019 -> 1788845019 00000004:00010000:2.0:1788845019.751818:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:2.0:1788845019.751820:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845019 -> 1788845019 00000004:00010000:2.0:1788845019.751822:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:2.0:1788845019.752567:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2367:x1875739641405184:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 1017us (1254us total) trans 4294967567 rc 0/0 00000100:00100000:2.0:1788845019.752578:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 348 00000100:00100000:2.0:1788845019.752638:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008917050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:2370:x1875739641405056:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 14026us (15492us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845019.752647:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 236 00000100:00100000:3.0:1788845019.756276:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.756445:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.756864:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641405312 00000100:00100000:0.0:1788845019.756880:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641405312 00000100:00100000:0.0:1788845019.756989:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 652 00000100:00100000:0.0:1788845019.757003:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+22:2368:x1875739641405312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.757027:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.757049:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe89 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78f1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.757097:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.757140:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660180736:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845019.757230:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.757809:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641405440 00000100:00100000:2.0:1788845019.757815:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641405440 00000100:00100000:2.0:1788845019.757855:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 415 00000100:00100000:2.0:1788845019.757863:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641405440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.757872:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.757914:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287abe90 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78f8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.757965:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660180864:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845019.758008:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.761017:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660180736:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.761049:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.761070:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe89 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78f1 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.761117:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.761120:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.761126:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ae00/0xb2187301287abe89 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78f1 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.761140:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251ae00, rc 0) 00000100:00100000:3.0:1788845019.761186:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660180864:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845019.761231:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287abe90 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78f8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.761264:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+22:2368:x1875739641405312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4263us (5001us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.761277:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 652 00010000:00010000:3.0:1788845019.761393:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287abe90 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78f8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845019.761420:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864000, rc 0) 00000100:00100000:3.0:1788845019.761489:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641405440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3626us (5043us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845019.761499:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 415 00000100:00100000:0.0:1788845019.768330:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.769788:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.770005:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641405568 00000100:00100000:2.0:1788845019.770019:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641405568 00000100:00100000:2.0:1788845019.770089:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 653 00000100:00100000:3.0:1788845019.770092:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641405696 00000100:00100000:3.0:1788845019.770103:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641405696 00000100:00100000:2.0:1788845019.770103:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:2368:x1875739641405568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.770128:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.770149:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abe97 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a78ff expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.770202:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.770250:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660180992:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845019.770272:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 416 00000100:00100000:3.0:1788845019.770288:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641405696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845019.770309:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845019.770328:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abe9e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7906 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.770363:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845019.770387:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a49180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660181120:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845019.770450:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d20f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.773494:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660180992:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845019.773523:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845019.773541:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abe97 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ff expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.773576:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.773580:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.773585:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abe97 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ff expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.773601:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845019.773668:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:2368:x1875739641405568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3569us (5346us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.773677:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 653 00000100:00100000:2.0:1788845019.774011:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a49180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660181120:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845019.774031:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abe9e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7906 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.774047:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287abe9e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7906 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.774056:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:2.0:1788845019.774093:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641405696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3807us (4314us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845019.774100:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 416 00000100:00100000:0.0:1788845019.779878:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845019.780829:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641405952 00000100:00100000:3.0:1788845019.780847:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641405952 00000100:00100000:0.0:1788845019.781451:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 654 00000100:00100000:0.0:1788845019.781472:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641405952:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845019.781533:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845019.781564:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287abe12 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a787a expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.781584:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.781589:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845019.781801:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.781817:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.781826:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.781861:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.781880:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 23 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.782424:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660181248:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845019.782522:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845019.796741:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660181248:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845019.796772:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.800388:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.801069:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641406080 00000100:00100000:1.0:1788845019.801091:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641406080 00010000:00010000:1.0:1788845019.801135:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 24 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.801311:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 237 00000100:00100000:1.0:1788845019.801322:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641406080:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845019.801337:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845019.801345:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 24 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.801363:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 24 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.801405:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a000/0xb2187301287abdc5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7834 expref: 24 pid: 5841 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.801422:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845019.801427:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.801437:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.801444:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.801476:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845019.801540:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641406080:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 220us (1162us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845019.801547:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 237 00010000:00010000:0.0:1788845019.801608:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.801615:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845019.801724:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967568 00000004:00010000:0.0:1788845019.801775:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788845019.801779:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845019 -> 1788845019 00000004:00010000:0.0:1788845019.801782:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:0.0:1788845019.801785:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845019 -> 1788845019 00000004:00010000:0.0:1788845019.801787:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:0.0:1788845019.801794:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000046d75a71 reply state 000000002c37cdcb transno 4294967569 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.801802:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845019.801878:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641405952:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 20415us (22008us total) trans 4294967569 rc 0/0 00000100:00100000:0.0:1788845019.801922:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 654 00000020:00080000:0.0:1788845019.801926:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845019.808603:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.808623:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.808631:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.808659:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0251aa00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845019.808665:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.808672:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.808691:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.808694:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845019.808699:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.808705:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251aa00/0xb2187301287abea5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.812313:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.812457:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641406208 00000100:00100000:2.0:1788845019.812469:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641406208 00000100:00100000:2.0:1788845019.812541:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 655 00000100:00100000:2.0:1788845019.812552:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641406208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.812596:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.812618:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a790d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.812648:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.812658:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.812690:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.812711:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 22 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.812724:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660181376:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845019.812800:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845019.820941:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.821131:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641406336 00000100:00100000:1.0:1788845019.821146:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641406336 00010000:00010000:1.0:1788845019.821200:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.821241:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 238 00000100:00100000:1.0:1788845019.821251:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641406336:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845019.821265:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845019.821288:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.821326:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.821348:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845019.821355:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.821366:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.821375:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.821413:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.821427:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660181504:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845019.821516:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788845019.821704:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b800/0xb2187301287abe82 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a78ea expref: 23 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.821720:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660181376:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845019.821748:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.821760:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.821768:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845019.821771:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845019.821797:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641406208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9248us (9495us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845019.821804:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 655 00000100:00100000:1.0:1788845019.831355:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660181504:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845019.831393:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845019.831452:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.831470:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641406336:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10216us (10587us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845019.831493:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 238 00000100:00100000:2.0:1788845019.832219:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641406464 00000100:00100000:2.0:1788845019.832229:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641406464 00000100:00100000:2.0:1788845019.832285:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 417 00000100:00100000:2.0:1788845019.832295:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641406464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.832313:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.832332:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7914 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.832365:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.832374:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(2 10) 00010000:00010000:2.0:1788845019.832379:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 10) 00010000:00010000:2.0:1788845019.832385:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.832415:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.832434:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 5 pid: 6685 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.832468:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660181632:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845019.832537:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845019.841581:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660181632:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845019.841623:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.841645:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864200, rc 0) 00000100:00100000:2.0:1788845019.841718:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641406464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9424us (10275us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845019.841728:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 417 00000100:00100000:0.0:1788845019.854505:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.856347:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641406592 00000100:00100000:2.0:1788845019.856363:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641406592 00010000:00010000:2.0:1788845019.856416:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 6 pid: 6685 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.856464:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 239 00000100:00100000:2.0:1788845019.856473:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641406592:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845019.856490:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845019.856524:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 6 pid: 6685 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.856549:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 6 pid: 6685 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.856595:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287abdf6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1183744->1187839) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7865 expref: 6 pid: 6685 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.856612:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 10) 00010000:00010000:2.0:1788845019.856616:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 10) 00010000:00010000:2.0:1788845019.856625:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.856634:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.856669:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845019.856683:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660181760:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845019.856800:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845019.864479:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.864543:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.864553:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660181760:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845019.864586:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845019.864666:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641406592:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8192us (10177us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845019.864676:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 239 00000100:00100000:3.0:1788845019.865631:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641406848 00000100:00100000:3.0:1788845019.865646:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641406848 00002000:00010000:3.0:1788845019.865704:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845019.865710:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02367800 with x1875739641406848 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845019.865727:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02367800. 00000100:00100000:3.0:1788845019.865816:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 249 00000100:00100000:3.0:1788845019.865828:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641406848:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845019.866625:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1404972 -> 0 00002000:00010000:3.0:1788845019.866631:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845019 -> 1788845019 00002000:00010000:3.0:1788845019.866634:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845019 -> 1788845018 00002000:00010000:3.0:1788845019.866636:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845019 -> 1788845019 00002000:00010000:3.0:1788845019.866638:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 373 -> 341 00000100:00100000:3.0:1788845019.866718:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641406848:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 888us (2172us total) trans 4294967375 rc 0/0 00000100:00100000:3.0:1788845019.866728:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 249 00002000:00010000:3.0:1788845019.866734:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845019.866737:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02367800 with x1875739641406848 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845019.866745:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02367800. 00000100:00100000:3.0:1788845019.866785:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641406720 00000100:00100000:3.0:1788845019.866792:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641406720 00002000:00010000:3.0:1788845019.866822:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845019.866827:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364e00 with x1875739641406720 00000100:00100000:3.0:1788845019.866863:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 349 00000100:00100000:3.0:1788845019.866872:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641406720:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845019.867182:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 696958 00000004:00010000:3.0:1788845019.867186:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845019 -> 1788845019 00000004:00010000:3.0:1788845019.867189:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:3.0:1788845019.867191:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845019 -> 1788845019 00000004:00010000:3.0:1788845019.867193:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845019.867228:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641406720:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 357us (2768us total) trans 4294967570 rc 0/0 00000100:00100000:3.0:1788845019.867245:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 349 00002000:00010000:3.0:1788845019.867249:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845019.867252:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364e00 with x1875739641406720 00000100:00100000:0.0:1788845019.881851:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.882250:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641406976 00000100:00100000:2.0:1788845019.882265:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641406976 00000100:00100000:2.0:1788845019.882522:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 656 00000100:00100000:2.0:1788845019.882552:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641406976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845019.882604:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845019.882634:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abeba lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a791b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.882764:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abec1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.882783:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abec1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.882797:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abec1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845019.882835:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 696958 00000004:00200000:2.0:1788845019.882840:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845019.882864:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abec1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.883491:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287abeba lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a791b expref: 27 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.883520:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abec1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a791b expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845019.883529:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845019.883533:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845019.883537:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845019.883818:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:14891:x1875739641406976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1267us (1980us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845019.883845:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 656 00000100:00100000:0.0:1788845019.893221:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.893497:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641407232 00000100:00100000:1.0:1788845019.893510:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641407232 00000100:00100000:1.0:1788845019.893616:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 418 00000100:00100000:1.0:1788845019.893628:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641407232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845019.893654:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845019.893678:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287abec8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7929 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.894055:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660181888:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845019.894281:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845019.897483:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.899732:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641407104 00000100:00100000:0.0:1788845019.899765:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641407104 00000100:00100000:0.0:1788845019.899866:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 657 00000100:00100000:0.0:1788845019.899880:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2367:x1875739641407104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.900329:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.900361:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abecf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7922 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.900438:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.900493:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660182016:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845019.900613:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f300 going to sleep for 11 seconds 00002000:00010000:1.0:1788845019.900996:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845018 -> 1788845019 00000100:00100000:1.0:1788845019.901029:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660181888:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845019.901475:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287abec8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7929 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.901500:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287abec8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7929 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.901514:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b600, rc 0) 00000100:00100000:1.0:1788845019.901573:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641407232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7948us (8367us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845019.901582:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 418 00000100:00100000:0.0:1788845019.906217:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660182016:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845019.906260:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 696958 00010000:00010000:0.0:1788845019.906305:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abecf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7922 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.906334:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.906338:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845019.906344:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abecf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7922 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.906364:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a400, rc 0) 00000100:00100000:0.0:1788845019.906458:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2367:x1875739641407104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6581us (8985us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845019.906469:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 657 00000100:00100000:3.0:1788845019.919559:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.920084:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641407360 00000100:00100000:0.0:1788845019.920095:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641407360 00000100:00100000:0.0:1788845019.920181:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 658 00000100:00100000:0.0:1788845019.920202:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:14891:x1875739641407360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845019.920224:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845019.920245:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7930 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.920323:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.920334:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.920368:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.920385:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 25 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845019.920400:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660182144:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845019.920534:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845019.927484:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660182144:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845019.927524:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.927545:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845019.927548:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845019.927552:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845019.927564:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a400, rc 0) 00000100:00100000:0.0:1788845019.927640:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641407360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7450us (8092us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845019.927664:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 658 00000100:00100000:3.0:1788845019.936091:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.936545:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641407488 00000100:00100000:1.0:1788845019.936565:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641407488 00010000:00010000:1.0:1788845019.936626:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 26 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.936829:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 240 00000100:00100000:1.0:1788845019.936989:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641407488:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845019.937006:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845019.937032:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 26 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.937052:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 26 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.937089:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287abeac lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a790d expref: 26 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.937107:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845019.937113:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.937125:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.937135:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845019.937173:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845019.937186:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660182272:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845019.937265:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845019.945098:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660182272:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845019.945147:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845019.945374:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641407488:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8387us (9308us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845019.945387:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 240 00000100:00100000:2.0:1788845019.952274:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845019.952463:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641407616 00000100:00100000:1.0:1788845019.952473:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641407616 00002000:00100000:1.0:1788845019.952525:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8700 x1875739641407616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845019.952537:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8700 with x1875739641407616 00000100:00100000:1.0:1788845019.952738:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 350 00000100:00100000:1.0:1788845019.952750:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641407616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845019.953064:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641407616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 316us (799us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845019.953080:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 350 00002000:00100000:1.0:1788845019.953088:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8700 x1875739641407616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845019.953097:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8700 with x1875739641407616 00000100:00100000:3.0:1788845019.961357:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.961527:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641407744 00000100:00100000:2.0:1788845019.961538:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641407744 00002000:00100000:2.0:1788845019.961599:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0199a680 x1875739641407744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.961611:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0199a680 with x1875739641407744 00000100:00100000:2.0:1788845019.961648:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 351 00000100:00100000:2.0:1788845019.961658:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0199a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641407744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.961960:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0199a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641407744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 300us (611us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.961968:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 351 00002000:00100000:2.0:1788845019.961975:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0199a680 x1875739641407744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.961983:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0199a680 with x1875739641407744 00000100:00100000:3.0:1788845019.972791:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.973105:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641407872 00000100:00100000:0.0:1788845019.973116:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641407872 00002000:00100000:0.0:1788845019.973170:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0199bb80 x1875739641407872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.973182:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0199bb80 with x1875739641407872 00000100:00100000:0.0:1788845019.973228:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 352 00000100:00100000:0.0:1788845019.973238:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0199bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641407872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.973515:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0199bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641407872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (731us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.973523:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 352 00002000:00100000:0.0:1788845019.973530:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0199bb80 x1875739641407872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.973538:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0199bb80 with x1875739641407872 00000100:00100000:0.0:1788845019.980603:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.981687:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641408000 00000100:00100000:2.0:1788845019.981700:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641408000 00002000:00100000:2.0:1788845019.981752:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ca80 x1875739641408000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.981763:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ca80 with x1875739641408000 00000100:00100000:2.0:1788845019.981805:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 353 00000100:00100000:2.0:1788845019.981816:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641408000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.982148:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641408000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 334us (1569us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.982173:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 353 00002000:00100000:2.0:1788845019.982182:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ca80 x1875739641408000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.982202:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ca80 with x1875739641408000 00000100:00100000:2.0:1788845019.987865:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845019.988008:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641408128 00000100:00100000:0.0:1788845019.988018:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641408128 00002000:00100000:0.0:1788845019.988081:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8380 x1875739641408128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.988092:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8380 with x1875739641408128 00000100:00100000:0.0:1788845019.988121:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 354 00000100:00100000:0.0:1788845019.988131:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641408128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845019.988536:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641408128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 405us (691us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845019.988564:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 354 00002000:00100000:0.0:1788845019.988573:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8380 x1875739641408128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845019.988581:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8380 with x1875739641408128 00000100:00100000:0.0:1788845019.993677:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845019.994336:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641408256 00000100:00100000:2.0:1788845019.994348:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641408256 00002000:00100000:2.0:1788845019.994398:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251df80 x1875739641408256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/0 e 0 to 0 dl 1788845030 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.994409:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251df80 with x1875739641408256 00000100:00100000:2.0:1788845019.994446:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 355 00000100:00100000:2.0:1788845019.994456:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641408256:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845019.994688:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641408256:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 232us (1019us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845019.994697:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 355 00002000:00100000:2.0:1788845019.994704:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251df80 x1875739641408256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:125/0 lens 488/4536 e 0 to 0 dl 1788845030 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845019.994711:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251df80 with x1875739641408256 00000100:00100000:2.0:1788845020.001214:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.001420:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641408384 00000100:00100000:0.0:1788845020.001444:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641408384 00002000:00100000:0.0:1788845020.001499:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fb480 x1875739641408384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.001511:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fb480 with x1875739641408384 00000100:00100000:0.0:1788845020.001552:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 356 00000100:00100000:0.0:1788845020.001562:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641408384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845020.001803:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641408384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 242us (612us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845020.001811:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 356 00002000:00100000:0.0:1788845020.001817:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fb480 x1875739641408384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.001825:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fb480 with x1875739641408384 00000100:00100000:0.0:1788845020.008273:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.008510:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641408512 00000100:00100000:2.0:1788845020.008522:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641408512 00002000:00100000:2.0:1788845020.008577:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251fb80 x1875739641408512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.008603:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251fb80 with x1875739641408512 00000100:00100000:2.0:1788845020.008670:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 357 00000100:00100000:2.0:1788845020.008680:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641408512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845020.008963:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641408512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 283us (700us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845020.008972:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 357 00002000:00100000:2.0:1788845020.008979:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251fb80 x1875739641408512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.008987:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251fb80 with x1875739641408512 00000100:00100000:3.0:1788845020.016186:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.016447:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641408640 00000100:00100000:0.0:1788845020.016457:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641408640 00002000:00100000:0.0:1788845020.016508:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01998700 x1875739641408640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.016519:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01998700 with x1875739641408640 00000100:00100000:0.0:1788845020.016623:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 358 00000100:00100000:0.0:1788845020.016633:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01998700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641408640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845020.016851:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01998700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641408640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 211us (666us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845020.016858:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 358 00002000:00100000:0.0:1788845020.016866:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01998700 x1875739641408640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.016875:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01998700 with x1875739641408640 00000100:00100000:1.0:1788845020.021685:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.022128:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641408768 00000100:00100000:3.0:1788845020.022139:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641408768 00002000:00100000:3.0:1788845020.022188:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c3800 x1875739641408768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.022212:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c3800 with x1875739641408768 00000100:00100000:3.0:1788845020.022252:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 359 00000100:00100000:3.0:1788845020.022263:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641408768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845020.022488:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641408768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 226us (812us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845020.022496:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 359 00002000:00100000:3.0:1788845020.022503:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c3800 x1875739641408768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.022511:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c3800 with x1875739641408768 00000100:00100000:2.0:1788845020.028649:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.028752:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641408896 00000100:00100000:1.0:1788845020.028762:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641408896 00002000:00100000:1.0:1788845020.028816:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9f80 x1875739641408896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.028827:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9f80 with x1875739641408896 00000100:00100000:1.0:1788845020.028871:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 360 00000100:00100000:1.0:1788845020.028879:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641408896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.029180:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641408896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 300us (539us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.029188:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 360 00002000:00100000:1.0:1788845020.029501:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9f80 x1875739641408896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.029510:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9f80 with x1875739641408896 00000100:00100000:3.0:1788845020.034240:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.035245:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641409024 00000100:00100000:1.0:1788845020.035257:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641409024 00002000:00100000:1.0:1788845020.035326:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0199b100 x1875739641409024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.035337:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0199b100 with x1875739641409024 00000100:00100000:1.0:1788845020.035367:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 361 00000100:00100000:1.0:1788845020.035378:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0199b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641409024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.035607:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0199b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641409024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (1375us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.035615:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 361 00002000:00100000:1.0:1788845020.035622:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0199b100 x1875739641409024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.035630:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0199b100 with x1875739641409024 00000100:00100000:2.0:1788845020.041842:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.042152:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641409152 00000100:00100000:1.0:1788845020.042164:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641409152 00002000:00100000:1.0:1788845020.042231:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fb800 x1875739641409152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.042243:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fb800 with x1875739641409152 00000100:00100000:1.0:1788845020.042299:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 362 00000100:00100000:1.0:1788845020.042310:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641409152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.042682:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641409152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 373us (848us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.042690:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 362 00002000:00100000:1.0:1788845020.042697:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fb800 x1875739641409152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.042706:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fb800 with x1875739641409152 00000100:00100000:2.0:1788845020.055257:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.055951:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641409280 00000100:00100000:1.0:1788845020.055961:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641409280 00002000:00100000:1.0:1788845020.056014:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa300 x1875739641409280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.056025:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa300 with x1875739641409280 00000100:00100000:1.0:1788845020.056062:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 363 00000100:00100000:1.0:1788845020.056083:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641409280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.056336:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641409280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 265us (1088us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.056344:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 363 00002000:00100000:1.0:1788845020.056351:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa300 x1875739641409280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.056359:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa300 with x1875739641409280 00000100:00100000:0.0:1788845020.065670:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.066289:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641409408 00000100:00100000:1.0:1788845020.066300:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641409408 00002000:00100000:1.0:1788845020.066352:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ea00 x1875739641409408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.066382:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ea00 with x1875739641409408 00000100:00100000:1.0:1788845020.066414:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 364 00000100:00100000:1.0:1788845020.066421:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641409408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.066635:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641409408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 213us (977us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.066647:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 364 00002000:00100000:1.0:1788845020.066655:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ea00 x1875739641409408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.066662:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ea00 with x1875739641409408 00000100:00100000:2.0:1788845020.071653:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.073560:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641409536 00000100:00100000:1.0:1788845020.073574:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641409536 00002000:00100000:1.0:1788845020.073631:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fad80 x1875739641409536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.073643:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fad80 with x1875739641409536 00000100:00100000:1.0:1788845020.073688:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 365 00000100:00100000:1.0:1788845020.073698:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641409536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.073998:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641409536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 300us (2357us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.074006:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 365 00002000:00100000:1.0:1788845020.074013:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fad80 x1875739641409536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.074022:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fad80 with x1875739641409536 00000100:00100000:2.0:1788845020.081625:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.081838:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641409664 00000100:00100000:1.0:1788845020.081859:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641409664 00002000:00100000:1.0:1788845020.081968:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9180 x1875739641409664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.081979:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9180 with x1875739641409664 00000100:00100000:1.0:1788845020.082064:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 366 00000100:00100000:1.0:1788845020.082085:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641409664:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.082367:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641409664:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 284us (750us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.082376:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 366 00002000:00100000:1.0:1788845020.082384:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9180 x1875739641409664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.082391:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9180 with x1875739641409664 00000100:00100000:3.0:1788845020.093857:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.094414:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641409792 00000100:00100000:1.0:1788845020.094426:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641409792 00002000:00100000:1.0:1788845020.094476:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01999f80 x1875739641409792/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.094487:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01999f80 with x1875739641409792 00000100:00100000:1.0:1788845020.094569:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 367 00000100:00100000:1.0:1788845020.094580:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01999f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641409792:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845020.101409:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 696958 -> 696958 00000004:00010000:1.0:1788845020.101419:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845019 -> 1788845020 00000004:00010000:1.0:1788845020.101423:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:1.0:1788845020.101425:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845019 -> 1788845020 00000004:00010000:1.0:1788845020.101428:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845020.102218:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01999f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2369:x1875739641409792:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 7621us (8355us total) trans 4294967571 rc 0/0 00000100:00100000:1.0:1788845020.102240:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 367 00002000:00100000:1.0:1788845020.102253:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01999f80 x1875739641409792/t4294967571(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/448 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.102262:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01999f80 with x1875739641409792 00000100:00100000:0.0:1788845020.105175:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.105507:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641409920 00000100:00100000:2.0:1788845020.105517:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641409920 00000100:00100000:2.0:1788845020.105605:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 659 00000100:00100000:2.0:1788845020.105614:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2369:x1875739641409920:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845020.167021:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967569 is committed 00000001:00080000:3.0:1788845020.167053:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967559 is committed 00000001:00080000:3.0:1788845020.167063:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967571 is committed 00000001:00080000:3.0:1788845020.167102:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967570 is committed 00000100:00100000:1.0:1788845020.167283:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641409920:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 61664us (62110us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845020.167296:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 659 00000100:00100000:3.0:1788845020.170338:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.170454:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641410048 00000100:00100000:0.0:1788845020.170464:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641410048 00000100:00100000:0.0:1788845020.170545:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 660 00000100:00100000:0.0:1788845020.170556:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0199b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641410048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.170576:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.170596:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b000/0xb2187301287abedd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7937 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.170634:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.170672:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660182400:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845020.170760:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000004:00010000:0.0:1788845020.172337:0:5841:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 712 -> 840 00000100:00100000:0.0:1788845020.172364:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660182400:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845020.172402:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 696958 00010000:00010000:0.0:1788845020.172418:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b000/0xb2187301287abedd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7937 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.172437:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845020.172440:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845020.172446:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251b000/0xb2187301287abedd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7937 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.172461:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b000, rc 0) 00000100:00100000:0.0:1788845020.172514:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0199b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641410048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1960us (2187us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845020.172522:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 660 00000100:00100000:3.0:1788845020.174540:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.174612:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641410176 00000100:00100000:0.0:1788845020.174621:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641410176 00000100:00100000:0.0:1788845020.174735:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 661 00000100:00100000:0.0:1788845020.174744:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01999c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641410176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.174764:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.174783:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abee4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a793e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.174818:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.174857:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660182528:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845020.174968:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845020.176719:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660182528:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845020.176746:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 696958 00010000:00010000:1.0:1788845020.176764:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abee4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a793e expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.176785:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.176788:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845020.176793:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251be00/0xb2187301287abee4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a793e expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.176807:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251be00, rc 0) 00000100:00100000:1.0:1788845020.177178:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01999c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641410176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2434us (2644us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845020.177187:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 661 00000100:00100000:3.0:1788845020.179664:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.179740:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641410432 00000100:00100000:1.0:1788845020.179751:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641410432 00000100:00100000:1.0:1788845020.179851:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 662 00000100:00100000:1.0:1788845020.179862:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01998e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641410432:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845020.179976:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845020.180002:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287abec1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a791b expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.180030:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.180035:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845020.180293:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.180308:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.180318:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845020.180367:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967572 00000004:00010000:1.0:1788845020.180394:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 696958 -> 696958 00000004:00010000:1.0:1788845020.180397:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845020 -> 1788845020 00000004:00010000:1.0:1788845020.180399:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:1.0:1788845020.180401:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845020 -> 1788845020 00000004:00010000:1.0:1788845020.180403:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 840 -> 840 00000004:00010000:1.0:1788845020.180410:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000251ad5f8 reply state 00000000a84c72a0 transno 4294967573 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.180418:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845020.180511:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01998e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641410432:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 650us (855us total) trans 4294967573 rc 0/0 00000100:00100000:1.0:1788845020.180518:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 662 00000020:00080000:1.0:1788845020.180523:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845020.182238:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.182252:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.182260:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.182266:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031e6b600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845020.182282:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.182290:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.182305:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.182308:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845020.182312:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.182319:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287abeeb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.182599:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.182798:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641410560 00000100:00100000:1.0:1788845020.182808:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641410560 00000100:00100000:1.0:1788845020.182862:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 663 00000100:00100000:1.0:1788845020.182872:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641410560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845020.182931:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.182957:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7945 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.182982:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.182990:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.183771:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.183791:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 9 pid: 5840 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.183806:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660182656:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845020.184082:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.187010:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.187306:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641410688 00000100:00100000:2.0:1788845020.187316:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641410688 00010000:00010000:2.0:1788845020.187354:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 10 pid: 5840 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.187400:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 241 00000100:00100000:2.0:1788845020.187410:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641410688:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845020.187424:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845020.187450:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 10 pid: 5840 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.187468:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 10 pid: 5840 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.187489:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.187496:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.187521:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.187530:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.187566:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.187579:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660182784:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845020.187660:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b900 going to sleep for 11 seconds 00010000:00010000:1.0:1788845020.187743:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a400/0xb2187301287abed6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7930 expref: 10 pid: 5840 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.187760:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660182656:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845020.187789:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.187809:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.187812:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845020.187815:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a000, rc 0) 00000100:00100000:1.0:1788845020.187862:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641410560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4991us (5266us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845020.187870:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 663 00000100:00100000:0.0:1788845020.191571:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.191618:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.192246:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641410816 00000100:00100000:3.0:1788845020.192255:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641410816 00002000:00010000:3.0:1788845020.192294:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845020.192299:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6680 with x1875739641410816 00000100:00100000:3.0:1788845020.192339:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 368 00000100:00100000:3.0:1788845020.192348:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641410816:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845020.192543:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 696958 -> 811433 00000004:00010000:3.0:1788845020.192547:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845020 -> 1788845020 00000004:00010000:3.0:1788845020.192550:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:3.0:1788845020.192552:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845020 -> 1788845020 00000004:00010000:3.0:1788845020.192554:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 840 -> 840 00000100:00100000:3.0:1788845020.192625:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641410816:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 278us (1064us total) trans 4294967574 rc 0/0 00000100:00100000:3.0:1788845020.192632:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 368 00002000:00010000:3.0:1788845020.192636:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845020.192638:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6680 with x1875739641410816 00000100:00100000:2.0:1788845020.194517:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660182784:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845020.194542:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845020.194602:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641410688:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7194us (7601us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.194613:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 241 00000100:00100000:1.0:1788845020.194941:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641410944 00000100:00100000:1.0:1788845020.194949:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641410944 00002000:00010000:1.0:1788845020.194986:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845020.194991:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7b80 with x1875739641410944 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845020.195001:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7b80. 00000100:00100000:1.0:1788845020.195038:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 250 00000100:00100000:1.0:1788845020.195049:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641410944:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845020.195282:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1788845020.195286:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845019 -> 1788845020 00002000:00010000:1.0:1788845020.195288:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845019 -> 1788845018 00002000:00010000:1.0:1788845020.195291:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845019 -> 1788845020 00002000:00010000:1.0:1788845020.195293:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 341 -> 341 00000100:00100000:1.0:1788845020.195335:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641410944:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 288us (3715us total) trans 4294967376 rc 0/0 00000100:00100000:1.0:1788845020.195342:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 250 00002000:00010000:1.0:1788845020.195345:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845020.195348:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7b80 with x1875739641410944 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845020.195352:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7b80. 00000100:00100000:3.0:1788845020.198208:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.198358:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641411072 00000100:00100000:0.0:1788845020.198369:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641411072 00000100:00100000:0.0:1788845020.198433:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 664 00000100:00100000:0.0:1788845020.198444:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01999880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641411072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.198465:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.198486:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251bc00/0xb2187301287abef9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a794c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.198572:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ac00/0xb2187301287abf00 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.198600:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ac00/0xb2187301287abf00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.198611:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ac00/0xb2187301287abf00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845020.198638:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 811433 00000004:00200000:0.0:1788845020.198643:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845020.198662:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ac00/0xb2187301287abf00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.198687:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251bc00/0xb2187301287abef9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a794c expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.198703:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ac00/0xb2187301287abf00 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a794c expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.198730:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845020.198733:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845020.198736:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251ac00, rc 0) 00000100:00100000:0.0:1788845020.198800:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01999880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641411072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 357us (599us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845020.198808:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 664 00000100:00100000:0.0:1788845020.203969:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.204014:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.204139:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641411328 00000100:00100000:3.0:1788845020.204150:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641411328 00000100:00100000:3.0:1788845020.204216:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 419 00000100:00100000:3.0:1788845020.204226:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641411328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.204261:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.204282:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287abf07 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a795a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.204348:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01998380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660182912:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.204433:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009dd40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845020.205256:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641411200 00000100:00100000:1.0:1788845020.205262:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641411200 00000100:00100000:1.0:1788845020.205311:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 665 00000100:00100000:1.0:1788845020.205320:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641411200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845020.205336:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.205352:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf0e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7953 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.205381:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.205407:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660183040:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845020.205465:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788845020.208128:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845018 -> 1788845019 00000100:00100000:3.0:1788845020.208157:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01998380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660182912:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.208193:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287abf07 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a795a expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.208216:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287abf07 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a795a expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.208229:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:3.0:1788845020.208288:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641411328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4063us (4328us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.208297:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 419 00000100:00100000:1.0:1788845020.208406:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660183040:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845020.208425:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:1.0:1788845020.208438:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf0e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7953 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.208451:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.208454:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845020.208458:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf0e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7953 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.208469:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b400, rc 0) 00000100:00100000:1.0:1788845020.208492:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641411200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3174us (4478us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845020.208499:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 665 00000100:00100000:3.0:1788845020.215601:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.216244:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641411456 00000100:00100000:0.0:1788845020.216256:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641411456 00000100:00100000:0.0:1788845020.216323:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 666 00000100:00100000:0.0:1788845020.216333:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01998380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641411456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.216355:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.216377:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251bc00/0xb2187301287abf15 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7961 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.216414:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.216454:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660183168:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845020.216544:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.223810:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.223969:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641411584 00000100:00100000:2.0:1788845020.223980:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641411584 00000100:00100000:2.0:1788845020.224055:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 420 00000100:00100000:2.0:1788845020.224074:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641411584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.224093:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.224112:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abf1c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7968 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.224170:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660183296:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845020.224278:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.229672:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660183168:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845020.229708:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:0.0:1788845020.229729:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251bc00/0xb2187301287abf15 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7961 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.229767:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845020.229770:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845020.229775:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251bc00/0xb2187301287abf15 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7961 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.229793:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251bc00, rc 0) 00000100:00100000:0.0:1788845020.229858:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01998380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641411456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13525us (14275us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845020.229866:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 666 00000100:00100000:2.0:1788845020.237220:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660183296:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845020.237264:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abf1c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7968 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.237288:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abf1c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7968 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.237300:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865400, rc 0) 00000100:00100000:2.0:1788845020.237568:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641411584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13503us (13766us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845020.237577:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 420 00000100:00100000:3.0:1788845020.244863:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.245137:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641411712 00000100:00100000:2.0:1788845020.245148:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641411712 00000100:00100000:2.0:1788845020.245335:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 667 00000100:00100000:2.0:1788845020.245347:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641411712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.245381:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.245400:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287abf23 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a796f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.245487:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.245502:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.245513:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845020.245551:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 811433 00000004:00200000:2.0:1788845020.245555:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845020.245571:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.245598:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287abf23 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.245613:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.245621:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.245624:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845020.245628:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845020.245686:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641411712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 341us (831us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.245693:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 667 00000100:00100000:2.0:1788845020.255247:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.259736:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641411968 00000100:00100000:1.0:1788845020.259752:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641411968 00000100:00100000:1.0:1788845020.259826:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 668 00000100:00100000:1.0:1788845020.259836:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641411968:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845020.259927:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845020.259956:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251ac00/0xb2187301287abf00 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a794c expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.259974:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.259979:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845020.260206:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.260221:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.260230:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.260264:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.260282:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 17 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.260296:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660183424:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845020.260625:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.268341:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.268496:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641412096 00000100:00100000:0.0:1788845020.268508:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641412096 00010000:00010000:0.0:1788845020.268552:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 18 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.268605:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 242 00000100:00100000:0.0:1788845020.268615:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641412096:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845020.268630:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845020.268636:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 18 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.268654:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 18 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.268690:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845020.268696:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.268720:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.268727:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.268746:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845020.268803:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641412096:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 189us (472us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845020.268811:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 242 00010000:00010000:1.0:1788845020.269281:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287abf2a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a796f expref: 17 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.269309:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660183424:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845020.269333:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.269341:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.269347:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845020.269468:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967575 00000004:00010000:1.0:1788845020.269522:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 811433 -> 811433 00000004:00010000:1.0:1788845020.269526:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845020 -> 1788845020 00000004:00010000:1.0:1788845020.269529:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:1.0:1788845020.269532:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845020 -> 1788845020 00000004:00010000:1.0:1788845020.269534:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 840 -> 840 00000004:00010000:1.0:1788845020.269543:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001faeb3f1 reply state 00000000d0307f21 transno 4294967576 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.269552:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845020.269641:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641411968:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 9806us (14404us total) trans 4294967576 rc 0/0 00000100:00100000:1.0:1788845020.269649:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 668 00000020:00080000:1.0:1788845020.269653:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845020.273255:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845020.273338:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.273350:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.273358:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.273365:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031e6b400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845020.273370:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.273378:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.273395:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845020.273398:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845020.273403:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.273427:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287abf31 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.273515:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641412224 00000100:00100000:1.0:1788845020.273524:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641412224 00000100:00100000:1.0:1788845020.273576:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 669 00000100:00100000:1.0:1788845020.273586:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036ce680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641412224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845020.273617:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.273634:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7976 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.273659:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.273668:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.274006:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.274023:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 16 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.274038:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660183552:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845020.274127:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845020.276349:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660183552:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845020.276386:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.276406:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.276409:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845020.276413:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.276440:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a200, rc 0) 00000100:00100000:1.0:1788845020.287258:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036ce680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641412224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13662us (14003us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845020.287272:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 669 00000100:00100000:3.0:1788845020.294003:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.294819:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641412352 00000100:00100000:0.0:1788845020.294830:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641412352 00010000:00010000:0.0:1788845020.294873:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 17 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.294940:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 243 00000100:00100000:0.0:1788845020.294951:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008917c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641412352:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845020.294967:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845020.294993:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 17 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.295012:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 17 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.295034:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abef2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7945 expref: 17 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.295052:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845020.295059:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.295081:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.295113:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.295162:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.295177:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660183680:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845020.295273:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.298322:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.298424:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660183680:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845020.298453:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845020.298514:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008917c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641412352:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3564us (4520us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.298526:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 243 00000100:00100000:3.0:1788845020.299026:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641412480 00000100:00100000:3.0:1788845020.299034:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641412480 00000100:00100000:3.0:1788845020.299111:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 421 00000100:00100000:3.0:1788845020.299121:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641412480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.299139:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.299156:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a797d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.299186:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.299195:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 10) 00010000:00010000:3.0:1788845020.299199:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 10) 00010000:00010000:3.0:1788845020.299205:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.299235:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.299253:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 7 pid: 6682 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.299266:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036cfb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660183808:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845020.299331:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009dd40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.306755:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.306850:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641412608 00000100:00100000:2.0:1788845020.306860:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641412608 00010000:00010000:2.0:1788845020.306927:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 8 pid: 6682 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.306997:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 244 00000100:00100000:2.0:1788845020.307007:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641412608:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845020.307022:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845020.307047:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 8 pid: 6682 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.307077:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 8 pid: 6682 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.307102:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 10) 00010000:00010000:2.0:1788845020.307107:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 10) 00010000:00010000:2.0:1788845020.307116:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.307127:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.307167:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.307182:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a4bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660183936:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845020.307266:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00010000:00010000:3.0:1788845020.307465:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287abeb3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7914 expref: 8 pid: 6682 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.307481:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036cfb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660183808:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845020.307516:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.307526:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:3.0:1788845020.307555:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641412480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8437us (9244us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845020.307561:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 421 00000100:00100000:0.0:1788845020.317657:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.317728:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641412736 00000100:00100000:3.0:1788845020.317736:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641412736 00002000:00010000:3.0:1788845020.317770:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1270398->18446744073709551615) 00002000:00100000:3.0:1788845020.317773:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7480 with x1875739641412736 ext(1270398->18446744073709551615) 00002000:00010000:3.0:1788845020.317783:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7480. 00000100:00100000:3.0:1788845020.317810:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 251 00000100:00100000:3.0:1788845020.317817:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641412736:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845020.318046:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1270398 00002000:00010000:3.0:1788845020.318050:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845020 -> 1788845020 00002000:00010000:3.0:1788845020.318052:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845019 -> 1788845018 00002000:00010000:3.0:1788845020.318055:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845020 -> 1788845020 00002000:00010000:3.0:1788845020.318057:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 341 -> 341 00000100:00100000:3.0:1788845020.318122:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641412736:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 304us (474us total) trans 4294967377 rc 0/0 00000100:00100000:3.0:1788845020.318130:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 251 00002000:00010000:3.0:1788845020.318133:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1270398->18446744073709551615) 00002000:00100000:3.0:1788845020.318136:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7480 with x1875739641412736 ext(1270398->18446744073709551615) 00002000:00010000:3.0:1788845020.318140:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7480. 00000100:00100000:2.0:1788845020.318158:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a4bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660183936:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845020.318194:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845020.318224:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641412608:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11219us (11480us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.318232:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 244 00000100:00100000:0.0:1788845020.320840:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.321231:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641412864 00000100:00100000:2.0:1788845020.321241:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641412864 00000100:00100000:2.0:1788845020.321302:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 670 00000100:00100000:2.0:1788845020.321311:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641412864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.321334:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.321357:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287abf46 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7984 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.321454:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abf4d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.321470:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abf4d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.321482:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abf4d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845020.321515:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1270398 00000004:00200000:2.0:1788845020.321520:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845020.321539:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abf4d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.321566:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287abf46 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7984 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.321768:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abf4d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7984 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.321793:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.321796:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845020.321799:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:2.0:1788845020.321954:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641412864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 644us (1122us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.321962:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 670 00000100:00100000:0.0:1788845020.325543:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.325687:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641412992 00000100:00100000:1.0:1788845020.325698:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641412992 00000100:00100000:1.0:1788845020.325760:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 671 00000100:00100000:1.0:1788845020.325772:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641412992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845020.325800:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.325823:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287abf54 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a798b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.326046:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.326072:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.326084:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845020.326112:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1270398 00000004:00200000:1.0:1788845020.326117:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845020.326137:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.326164:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287abf54 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.326202:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.326210:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.326213:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845020.326216:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b000, rc 0) 00000100:00100000:1.0:1788845020.326279:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641412992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 508us (745us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845020.326285:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 671 00000100:00100000:3.0:1788845020.328342:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.328642:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641413120 00000100:00100000:1.0:1788845020.328652:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641413120 00000100:00100000:3.0:1788845020.329567:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.329605:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641413248 00000100:00100000:3.0:1788845020.329610:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641413248 00000100:00100000:3.0:1788845020.329647:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 422 00000100:00100000:3.0:1788845020.329656:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641413248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.329674:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.329689:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287abf62 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7999 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.329745:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036ced80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660184064:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.329818:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009dd40c0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845020.331817:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845018 -> 1788845019 00000100:00100000:2.0:1788845020.331845:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036ced80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660184064:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845020.331902:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287abf62 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7999 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.331928:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287abf62 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7999 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.331951:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:2.0:1788845020.332001:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641413248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2345us (2437us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845020.332009:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 422 00000100:00100000:1.0:1788845020.332905:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 672 00000100:00100000:1.0:1788845020.332924:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641413120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845020.332955:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.332974:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abf69 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7992 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.333007:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.333046:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660184192:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845020.333175:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845020.340535:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660184192:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845020.340566:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:1.0:1788845020.340597:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abf69 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7992 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.340632:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.340636:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845020.340642:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abf69 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7992 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.340657:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a800, rc 0) 00000100:00100000:1.0:1788845020.342298:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641413120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9373us (13955us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845020.342309:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 672 00000100:00100000:2.0:1788845020.345475:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.345552:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.345578:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641413504 00000100:00100000:2.0:1788845020.345598:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641413504 00000100:00100000:1.0:1788845020.345792:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641413376 00000100:00100000:1.0:1788845020.345796:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641413376 00000100:00100000:1.0:1788845020.345838:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 673 00000100:00100000:1.0:1788845020.345848:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641413376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845020.345868:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.349716:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abf70 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79a0 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.349773:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.349818:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660184320:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845020.349929:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845020.353546:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660184320:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845020.353578:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:1.0:1788845020.353721:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abf70 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79a0 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.353761:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.353764:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845020.353770:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a000/0xb2187301287abf70 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79a0 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.353786:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a000, rc 0) 00000100:00100000:1.0:1788845020.353848:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641413376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8001us (8382us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845020.353857:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 673 00000100:00100000:2.0:1788845020.356368:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 423 00000100:00100000:2.0:1788845020.356391:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641413504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.356422:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.356442:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abf77 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79a7 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.356508:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660184448:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845020.356609:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.370385:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660184704:0@lo:400:kworker.0 00000100:00100000:3.0:1788845020.370460:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845020.370494:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660185216:0@lo:400:kworker.0 00000100:00100000:3.0:1788845020.370518:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845020.370541:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845020.370546:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660184832:0@lo:400:kworker.0 00000100:00100000:3.0:1788845020.370567:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845020.370612:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660185216 00000100:00100000:3.0:1788845020.370620:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660185216 00000100:00100000:3.0:1788845020.370672:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:3.0:1788845020.370682:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660185216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845020.370732:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660185216:12345-0@lo:400:kworker.0 Request processed in 52us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845020.370739:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00080000:3.0:1788845020.370772:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845020.370804:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660185216:0@lo:400:kworker.0 00000100:00100000:3.0:1788845020.370827:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660184704 00000100:00100000:3.0:1788845020.370831:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660184704 00000100:00100000:3.0:1788845020.370855:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660184832 00000100:00100000:3.0:1788845020.370858:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660184832 00000100:00080000:3.0:1788845020.370871:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845020.370912:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:3.0:1788845020.370920:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660184704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845020.371054:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660184960:0@lo:400:kworker.0 00000100:00100000:2.0:1788845020.371134:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660184576:0@lo:400:kworker.0 00000100:00100000:1.0:1788845020.373593:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845020.375802:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845020.375855:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660185088:0@lo:400:kworker.0 00000100:00100000:2.0:1788845020.375907:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845020.375967:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660185088 00000100:00100000:2.0:1788845020.375974:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660185088 00000100:00100000:2.0:1788845020.376023:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:2.0:1788845020.376031:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660185088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845020.376088:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660185088:12345-0@lo:400:kworker.0 Request processed in 58us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.376095:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:2.0:1788845020.376130:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660185088:0@lo:400:kworker.0 00000100:00100000:1.0:1788845020.377794:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660184576 00000100:00100000:1.0:1788845020.377806:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660184576 00000100:00100000:1.0:1788845020.377851:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:1.0:1788845020.377858:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660184576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845020.377949:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660184576:12345-0@lo:400:kworker.0 Request processed in 91us (2152us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845020.377957:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:2.0:1788845020.378195:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660184576:0@lo:400:kworker.0 00000100:00100000:1.0:1788845020.379011:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:1.0:1788845020.379022:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660184832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845020.379095:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660184832:12345-0@lo:400:kworker.0 Request processed in 76us (8526us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845020.379102:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:3.0:1788845020.379731:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660184704:12345-0@lo:400:kworker.0 Request processed in 8809us (9268us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845020.379739:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:3.0:1788845020.379767:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660184960 00000100:00100000:3.0:1788845020.379774:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660184960 00000100:00100000:3.0:1788845020.379807:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:3.0:1788845020.379814:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660184960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845020.379866:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660184960:12345-0@lo:400:kworker.0 Request processed in 52us (6289us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845020.379872:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00100000:1.0:1788845020.380146:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660184960:0@lo:400:kworker.0 00000100:00080000:3.0:1788845020.380258:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1788845020.380266:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845020.380290:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660184832:0@lo:400:kworker.0 00000100:00100000:3.0:1788845020.380303:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660184704:0@lo:400:kworker.0 00000100:00100000:2.0:1788845020.382998:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660184448:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845020.383046:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abf77 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79a7 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.383085:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287abf77 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79a7 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.383113:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865400, rc 0) 00000100:00100000:2.0:1788845020.383168:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641413504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 26788us (37615us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845020.383178:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 423 00000100:00100000:0.0:1788845020.389533:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.389628:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641413632 00000100:00100000:2.0:1788845020.389639:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641413632 00002000:00100000:2.0:1788845020.389703:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efa680 x1875739641413632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.389714:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efa680 with x1875739641413632 00000100:00100000:2.0:1788845020.389751:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 369 00000100:00100000:2.0:1788845020.389761:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641413632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845020.390056:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641413632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 295us (532us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845020.390073:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 369 00002000:00100000:2.0:1788845020.390081:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efa680 x1875739641413632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.390089:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efa680 with x1875739641413632 00000100:00100000:3.0:1788845020.396531:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.397511:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641413760 00000100:00100000:2.0:1788845020.397522:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641413760 00002000:00100000:2.0:1788845020.397573:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cc380 x1875739641413760/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.397595:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cc380 with x1875739641413760 00000100:00100000:2.0:1788845020.397634:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 370 00000100:00100000:2.0:1788845020.397658:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641413760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845020.398089:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641413760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 431us (1568us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845020.398098:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 370 00002000:00100000:2.0:1788845020.398105:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cc380 x1875739641413760/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.398113:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cc380 with x1875739641413760 00000100:00100000:0.0:1788845020.402428:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.404727:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641413888 00000100:00100000:2.0:1788845020.404742:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641413888 00002000:00100000:2.0:1788845020.404802:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efb100 x1875739641413888/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.404815:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efb100 with x1875739641413888 00000100:00100000:2.0:1788845020.404855:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 371 00000100:00100000:2.0:1788845020.404864:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641413888:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845020.405947:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641413888:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1068us (3513us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845020.405958:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 371 00002000:00100000:2.0:1788845020.405967:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efb100 x1875739641413888/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.405975:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efb100 with x1875739641413888 00000100:00100000:3.0:1788845020.410510:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.410629:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641414016 00000100:00100000:2.0:1788845020.410639:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641414016 00002000:00100000:2.0:1788845020.410703:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cf800 x1875739641414016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.410736:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cf800 with x1875739641414016 00000100:00100000:2.0:1788845020.410778:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 372 00000100:00100000:2.0:1788845020.410788:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845020.411290:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 500us (787us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845020.411298:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 372 00002000:00100000:2.0:1788845020.411306:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cf800 x1875739641414016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.411314:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cf800 with x1875739641414016 00000100:00100000:3.0:1788845020.421330:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.421850:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641414144 00000100:00100000:1.0:1788845020.421861:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641414144 00002000:00100000:1.0:1788845020.422007:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cd500 x1875739641414144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.422017:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cd500 with x1875739641414144 00000100:00100000:1.0:1788845020.422057:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 373 00000100:00100000:1.0:1788845020.422077:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.422336:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (1015us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.422344:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 373 00002000:00100000:1.0:1788845020.422351:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cd500 x1875739641414144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.422359:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cd500 with x1875739641414144 00000100:00100000:1.0:1788845020.430746:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.430927:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641414272 00000100:00100000:0.0:1788845020.430948:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641414272 00002000:00100000:0.0:1788845020.431002:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359ae680 x1875739641414272/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.431014:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359ae680 with x1875739641414272 00000100:00100000:0.0:1788845020.431053:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 374 00000100:00100000:0.0:1788845020.431071:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845020.431335:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 273us (597us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845020.431343:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 374 00002000:00100000:0.0:1788845020.431359:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359ae680 x1875739641414272/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.431366:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359ae680 with x1875739641414272 00000100:00100000:3.0:1788845020.441178:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.441350:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641414400 00000100:00100000:1.0:1788845020.441362:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641414400 00002000:00100000:1.0:1788845020.441413:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cdc00 x1875739641414400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.441424:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cdc00 with x1875739641414400 00000100:00100000:1.0:1788845020.441460:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 375 00000100:00100000:1.0:1788845020.441470:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.441747:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (578us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.441755:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 375 00002000:00100000:1.0:1788845020.441769:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cdc00 x1875739641414400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.441777:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cdc00 with x1875739641414400 00000100:00100000:3.0:1788845020.447597:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.447810:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641414528 00000100:00100000:0.0:1788845020.447820:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641414528 00002000:00100000:0.0:1788845020.447873:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cc000 x1875739641414528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.447921:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cc000 with x1875739641414528 00000100:00100000:0.0:1788845020.447969:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 376 00000100:00100000:0.0:1788845020.447978:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414528:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845020.448239:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414528:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (663us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845020.448247:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 376 00002000:00100000:0.0:1788845020.448254:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cc000 x1875739641414528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.448262:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cc000 with x1875739641414528 00000100:00100000:3.0:1788845020.456427:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.456540:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641414656 00000100:00100000:1.0:1788845020.456551:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641414656 00002000:00100000:1.0:1788845020.456616:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036ce300 x1875739641414656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.456628:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036ce300 with x1875739641414656 00000100:00100000:1.0:1788845020.456669:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 377 00000100:00100000:1.0:1788845020.456680:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036ce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414656:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.457172:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036ce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641414656:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 493us (754us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.457181:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 377 00002000:00100000:1.0:1788845020.457189:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036ce300 x1875739641414656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.457197:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036ce300 with x1875739641414656 00000100:00100000:3.0:1788845020.478811:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.479118:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641414784 00000100:00100000:0.0:1788845020.479128:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641414784 00002000:00100000:0.0:1788845020.479175:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036ced80 x1875739641414784/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.479186:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036ced80 with x1875739641414784 00000100:00100000:0.0:1788845020.479292:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 378 00000100:00100000:0.0:1788845020.479304:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036ced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641414784:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845020.479597:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036ced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641414784:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 295us (793us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845020.479621:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 378 00002000:00100000:0.0:1788845020.479630:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036ced80 x1875739641414784/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845020.479638:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036ced80 with x1875739641414784 00000100:00100000:2.0:1788845020.500493:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.500611:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641414912 00000100:00100000:1.0:1788845020.500622:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641414912 00002000:00100000:1.0:1788845020.500673:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366d80 x1875739641414912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.500685:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366d80 with x1875739641414912 00000100:00100000:1.0:1788845020.500721:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 379 00000100:00100000:1.0:1788845020.500732:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641414912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.501032:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641414912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 301us (547us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.501057:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 379 00002000:00100000:1.0:1788845020.501075:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366d80 x1875739641414912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.501084:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366d80 with x1875739641414912 00000100:00100000:0.0:1788845020.511352:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.511763:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641415040 00000100:00100000:2.0:1788845020.511773:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641415040 00002000:00100000:2.0:1788845020.511824:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efb480 x1875739641415040/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.511836:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efb480 with x1875739641415040 00000100:00100000:2.0:1788845020.511875:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 380 00000100:00100000:2.0:1788845020.511913:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845020.512293:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 408us (951us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.512305:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 380 00002000:00100000:2.0:1788845020.512313:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efb480 x1875739641415040/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/440 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.512321:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efb480 with x1875739641415040 00000100:00100000:3.0:1788845020.527984:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.528453:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641415168 00000100:00100000:1.0:1788845020.528465:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641415168 00002000:00100000:1.0:1788845020.528516:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cf480 x1875739641415168/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.528528:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cf480 with x1875739641415168 00000100:00100000:1.0:1788845020.528570:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 381 00000100:00100000:1.0:1788845020.528591:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641415168:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.528844:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641415168:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 264us (869us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.528852:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 381 00002000:00100000:1.0:1788845020.528858:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f036cf480 x1875739641415168/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.528868:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f036cf480 with x1875739641415168 00000100:00100000:0.0:1788845020.533225:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.533398:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641415296 00000100:00100000:1.0:1788845020.533409:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641415296 00002000:00100000:1.0:1788845020.533461:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8380 x1875739641415296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.533472:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8380 with x1875739641415296 00000100:00100000:1.0:1788845020.533512:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 382 00000100:00100000:1.0:1788845020.533522:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.533960:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 438us (742us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.533969:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 382 00002000:00100000:1.0:1788845020.533977:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8380 x1875739641415296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.533985:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8380 with x1875739641415296 00000100:00100000:0.0:1788845020.539610:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.539776:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641415424 00000100:00100000:1.0:1788845020.539786:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641415424 00002000:00100000:1.0:1788845020.539830:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efb800 x1875739641415424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.539842:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efb800 with x1875739641415424 00000100:00100000:1.0:1788845020.539878:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 383 00000100:00100000:1.0:1788845020.539907:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.540320:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 413us (718us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.540328:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 383 00002000:00100000:1.0:1788845020.540335:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efb800 x1875739641415424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.540344:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efb800 with x1875739641415424 00000100:00100000:2.0:1788845020.545249:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.545347:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641415552 00000100:00100000:3.0:1788845020.545358:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641415552 00002000:00100000:3.0:1788845020.545405:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02360000 x1875739641415552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.545416:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02360000 with x1875739641415552 00000100:00100000:3.0:1788845020.545457:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 384 00000100:00100000:3.0:1788845020.545466:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641415552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845020.545841:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641415552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 375us (601us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845020.545849:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 384 00002000:00100000:3.0:1788845020.545855:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02360000 x1875739641415552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.545863:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02360000 with x1875739641415552 00000100:00100000:0.0:1788845020.550767:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.550966:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641415680 00000100:00100000:1.0:1788845020.550977:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641415680 00002000:00100000:1.0:1788845020.551029:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efa300 x1875739641415680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.551040:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efa300 with x1875739641415680 00000100:00100000:1.0:1788845020.551104:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 385 00000100:00100000:1.0:1788845020.551124:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035efa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.551385:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035efa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 272us (626us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.551392:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 385 00002000:00100000:1.0:1788845020.551399:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035efa300 x1875739641415680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.551406:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035efa300 with x1875739641415680 00000100:00100000:0.0:1788845020.556409:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.556760:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641415808 00000100:00100000:3.0:1788845020.556770:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641415808 00002000:00100000:3.0:1788845020.556816:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8700 x1875739641415808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.556827:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8700 with x1875739641415808 00000100:00100000:3.0:1788845020.556864:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 386 00000100:00100000:3.0:1788845020.556874:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641415808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845020.557186:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641415808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 311us (784us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845020.557208:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 386 00002000:00100000:3.0:1788845020.557215:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8700 x1875739641415808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.557223:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8700 with x1875739641415808 00000100:00100000:0.0:1788845020.561874:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.562375:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641415936 00000100:00100000:1.0:1788845020.562384:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641415936 00002000:00100000:1.0:1788845020.562428:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8a80 x1875739641415936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.562439:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8a80 with x1875739641415936 00000100:00100000:1.0:1788845020.562472:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 387 00000100:00100000:1.0:1788845020.562481:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.562865:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641415936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 384us (999us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845020.562873:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 387 00002000:00100000:1.0:1788845020.562880:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef8a80 x1875739641415936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.562905:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef8a80 with x1875739641415936 00000100:00100000:0.0:1788845020.567012:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.567312:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641416064 00000100:00100000:3.0:1788845020.567322:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641416064 00002000:00100000:3.0:1788845020.567372:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef9180 x1875739641416064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.567383:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef9180 with x1875739641416064 00000100:00100000:3.0:1788845020.567424:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 388 00000100:00100000:3.0:1788845020.567433:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035ef9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641416064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845020.567697:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035ef9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641416064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 264us (692us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845020.567705:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 388 00002000:00100000:3.0:1788845020.567712:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035ef9180 x1875739641416064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.567720:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035ef9180 with x1875739641416064 00000100:00100000:0.0:1788845020.572660:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.572762:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641416192 00000100:00100000:2.0:1788845020.572772:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641416192 00002000:00100000:2.0:1788845020.572822:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da49c00 x1875739641416192/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.572833:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da49c00 with x1875739641416192 00000100:00100000:2.0:1788845020.572872:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 389 00000100:00100000:2.0:1788845020.572922:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641416192:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845020.583303:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 811433 -> 811433 00000004:00010000:2.0:1788845020.583314:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845020 -> 1788845020 00000004:00010000:2.0:1788845020.583317:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:2.0:1788845020.583319:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845020 -> 1788845020 00000004:00010000:2.0:1788845020.583322:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 840 -> 840 00000100:00100000:2.0:1788845020.583427:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641416192:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 10542us (10850us total) trans 4294967577 rc 0/0 00000100:00100000:2.0:1788845020.583438:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 389 00002000:00100000:2.0:1788845020.583450:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da49c00 x1875739641416192/t4294967577(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/448 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.583460:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da49c00 with x1875739641416192 00000100:00100000:0.0:1788845020.592182:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.592545:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641416320 00000100:00100000:2.0:1788845020.592556:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641416320 00000100:00100000:2.0:1788845020.592791:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 674 00000100:00100000:2.0:1788845020.592803:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641416320:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:2.0:1788845020.630708:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641416320:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 37896us (38811us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.630740:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 674 00000001:00080000:3.0:1788845020.630917:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967576 is committed 00000001:00080000:3.0:1788845020.630931:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967573 is committed 00000001:00080000:3.0:1788845020.630956:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967577 is committed 00000001:00080000:3.0:1788845020.630964:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967574 is committed 00000100:00100000:0.0:1788845020.634728:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.634773:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.634855:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641416576 00000100:00100000:3.0:1788845020.634865:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641416576 00000100:00100000:3.0:1788845020.635068:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 424 00000100:00100000:3.0:1788845020.635080:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641416576:12345-192.168.201.29@tcp:101:fsx.0 00000100:00100000:2.0:1788845020.635130:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641416448 00000100:00100000:2.0:1788845020.635135:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641416448 00000100:00100000:2.0:1788845020.635167:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 675 00000100:00100000:2.0:1788845020.635175:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641416448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.635193:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.635213:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abf7e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79ae expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.635250:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.635290:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660185344:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845020.635376:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788845020.635378:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.635390:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abf85 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79b5 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.635423:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02486d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660185472:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.635449:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000004:00010000:2.0:1788845020.638950:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845019 -> 1788845020 00000004:00010000:2.0:1788845020.638960:0:5840:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 840 -> 968 00000100:00100000:2.0:1788845020.638986:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660185344:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845020.639012:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:2.0:1788845020.639030:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abf7e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79ae expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.639051:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.639054:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845020.639068:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287abf7e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79ae expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.639102:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00000100:00100000:2.0:1788845020.639166:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641416448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3991us (4447us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845020.639175:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 675 00002000:00010000:3.0:1788845020.642046:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845019 -> 1788845020 00000100:00100000:3.0:1788845020.642089:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02486d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660185472:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.642201:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abf85 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79b5 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.642225:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abf85 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79b5 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.642238:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845020.642348:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641416576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7269us (7572us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.642357:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 424 00000100:00100000:0.0:1788845020.647668:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.647794:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.647859:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641416832 00000100:00100000:2.0:1788845020.647868:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641416832 00000100:00100000:1.0:1788845020.651333:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641416704 00000100:00100000:1.0:1788845020.651350:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641416704 00000100:00100000:2.0:1788845020.651634:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 425 00000100:00100000:2.0:1788845020.651648:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641416832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.651671:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.651689:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287abf8c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79c3 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.651746:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660185600:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845020.651830:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.653929:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660185600:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.653973:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287abf8c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79c3 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.653997:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287abf8c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79c3 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.654026:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864400, rc 0) 00000100:00100000:3.0:1788845020.654134:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641416832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2489us (6338us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.654142:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 425 00000100:00100000:1.0:1788845020.654521:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 676 00000100:00100000:1.0:1788845020.654537:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641416704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845020.654563:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.654591:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abf93 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79bc expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.654627:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.654668:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359ae300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660185728:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845020.654716:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845020.657863:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359ae300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660185728:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845020.658085:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:1.0:1788845020.658181:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abf93 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79bc expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.658205:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845020.658209:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845020.658215:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abf93 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79bc expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.658231:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a800, rc 0) 00000100:00100000:1.0:1788845020.658396:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641416704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3864us (10736us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845020.658405:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 676 00000100:00100000:0.0:1788845020.669648:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.669731:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641416960 00000100:00100000:3.0:1788845020.669741:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641416960 00002000:00100000:3.0:1788845020.669791:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48a80 x1875739641416960/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 520/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.669802:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48a80 with x1875739641416960 00000100:00100000:3.0:1788845020.669838:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 390 00000100:00100000:3.0:1788845020.669847:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641416960:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845020.684372:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.684584:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641417088 00000100:00100000:2.0:1788845020.684595:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641417088 00002000:00100000:2.0:1788845020.684650:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1273855) req@ffff94a019d92a00 x1875739641417088/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.684662:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d92a00 with x1875739641417088 ext(1048576->1273855) 00002000:00010000:2.0:1788845020.684668:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d92a00 00000100:00100000:2.0:1788845020.684701:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 252 00000100:00100000:2.0:1788845020.684712:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641417088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845020.692785:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641416960:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 22929us (23138us total) trans 0 rc 737280/737280 00000100:00100000:3.0:1788845020.692800:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 390 00002000:00100000:3.0:1788845020.692814:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48a80 x1875739641416960/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 520/440 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 737280/737280 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.692825:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48a80 with x1875739641416960 00000100:00100000:1.0:1788845020.695692:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641417088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 10968us (11316us total) trans 0 rc 225280/225280 00000100:00100000:1.0:1788845020.695707:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 252 00002000:00100000:1.0:1788845020.695738:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1273855) req@ffff94a019d92a00 x1875739641417088/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/440 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 225280/225280 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.695749:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d92a00 with x1875739641417088 ext(1048576->1273855) 00002000:00010000:1.0:1788845020.695757:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d92a00 00000100:00100000:0.0:1788845020.708730:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.711408:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641417344 00000100:00100000:1.0:1788845020.711424:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641417344 00000100:00100000:1.0:1788845020.711547:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 38 00000100:00100000:1.0:1788845020.711558:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641417344:12345-192.168.201.29@tcp:35:fsx.0 00000100:00100000:1.0:1788845020.712009:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641417344:12345-192.168.201.29@tcp:35:fsx.0 Request processed in 452us (3289us total) trans 4294967578 rc 0/0 00000100:00100000:1.0:1788845020.712017:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 38 00000100:00100000:0.0:1788845020.717502:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.717603:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641417600 00000100:00100000:2.0:1788845020.717613:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641417600 00000100:00100000:2.0:1788845020.717736:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 677 00000100:00100000:2.0:1788845020.717747:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641417600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.717777:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.717800:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287abf9a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79ca expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.717876:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718016:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718029:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718096:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718104:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718111:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718410:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718426:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287abfaf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718436:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287abfaf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718447:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287abfaf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718453:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287abfaf lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718459:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031d3fc00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845020.718463:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287abfaf lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718470:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287abfaf lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718481:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.718485:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287abfaf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718491:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287abfaf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845020.718505:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1270398 00010000:00010000:2.0:1788845020.718580:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718587:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718592:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718597:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031d3f600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845020.718605:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718612:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718621:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:2.0:1788845020.718623:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845020.718627:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718633:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfa1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718642:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718648:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718653:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718658:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031d3e800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845020.718661:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718666:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718673:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3d6fced7 (ffff94a0087c6300) 00010000:00010000:2.0:1788845020.718675:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845020.718684:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718689:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287abfa8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718727:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287abf9a lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79ca expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718737:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:2.0:1788845020.718739:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788845020.718743:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287abf9a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79ca expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.718753:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:2.0:1788845020.718812:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641417600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1067us (1320us total) trans 4294967579 rc 301/301 00000100:00100000:2.0:1788845020.718819:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 677 00000100:00100000:2.0:1788845020.723185:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.725077:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641417728 00000100:00100000:1.0:1788845020.725091:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641417728 00000100:00100000:1.0:1788845020.725153:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 678 00000100:00100000:1.0:1788845020.725163:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641417728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845020.725188:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.725218:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79d1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.725248:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.725380:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.725418:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.725452:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 10 pid: 11036 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.725466:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359ae300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660185856:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845020.725607:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845020.738516:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.741630:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359ae300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660185856:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845020.741675:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.741697:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845020.741700:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845020.741723:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.741733:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a800, rc 0) 00000100:00100000:3.0:1788845020.741816:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641417728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16653us (18638us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845020.741825:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 678 00000100:00100000:2.0:1788845020.746449:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641417856 00000100:00100000:2.0:1788845020.746466:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641417856 00010000:00010000:2.0:1788845020.746519:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 11 pid: 11036 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.746677:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 245 00000100:00100000:2.0:1788845020.746689:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641417856:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845020.746704:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845020.746733:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 11 pid: 11036 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.746764:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 11 pid: 11036 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.746785:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287abf38 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7976 expref: 11 pid: 11036 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.746800:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.746806:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.746817:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.746826:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.746866:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.746907:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660185984:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845020.747124:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.754161:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.754189:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641417984 00000100:00100000:3.0:1788845020.754198:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641417984 00002000:00100000:3.0:1788845020.754245:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02486d80 x1875739641417984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.754256:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02486d80 with x1875739641417984 00000100:00100000:3.0:1788845020.754291:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 391 00000100:00100000:3.0:1788845020.754300:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02486d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641417984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845020.754555:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02486d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641417984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 255us (403us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845020.754563:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 391 00002000:00100000:3.0:1788845020.754570:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02486d80 x1875739641417984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.756196:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02486d80 with x1875739641417984 00000100:00100000:2.0:1788845020.756511:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660185984:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845020.756536:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845020.756669:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641417856:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9982us (18162us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.756697:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 245 00000100:00100000:1.0:1788845020.761995:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.762191:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641418112 00000100:00100000:3.0:1788845020.762201:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641418112 00002000:00100000:3.0:1788845020.762250:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359adf80 x1875739641418112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.762261:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359adf80 with x1875739641418112 00000100:00100000:3.0:1788845020.762313:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 392 00000100:00100000:3.0:1788845020.762323:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641418112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845020.762564:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641418112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 242us (578us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845020.762572:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 392 00002000:00100000:3.0:1788845020.762659:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359adf80 x1875739641418112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:126/0 lens 488/4536 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.762668:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359adf80 with x1875739641418112 00010000:00010000:0.0:1788845020.765840:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845020.768483:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.768795:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641418240 00000100:00100000:0.0:1788845020.768806:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641418240 00000100:00100000:0.0:1788845020.769331:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 426 00000100:00100000:0.0:1788845020.769345:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02484e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641418240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.769382:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.769403:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287abfc4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79d8 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.769465:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d93480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660186112:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845020.769545:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.771689:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d93480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660186112:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.771729:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287abfc4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d8 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.771752:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a9ffade5/0xb2187301287abfc4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d8 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.771765:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251b400, rc 0) 00000100:00100000:3.0:1788845020.771850:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02484e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641418240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2508us (3377us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.771859:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 426 00000100:00100000:1.0:1788845020.775759:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.775863:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641418368 00000100:00100000:0.0:1788845020.775873:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641418368 00000100:00100000:0.0:1788845020.775980:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 427 00000100:00100000:0.0:1788845020.775990:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641418368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.776007:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.776023:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000484d8821/0xb2187301287abfcb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79df expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.776094:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d92680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660186240:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845020.776171:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.779116:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d92680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660186240:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845020.779164:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000484d8821/0xb2187301287abfcb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79df expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.779190:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000484d8821/0xb2187301287abfcb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79df expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.779203:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a800, rc 0) 00000100:00100000:0.0:1788845020.779262:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641418368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3272us (3514us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845020.779278:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 427 00000100:00100000:1.0:1788845020.783173:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.783324:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641418496 00000100:00100000:3.0:1788845020.783333:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641418496 00000100:00100000:3.0:1788845020.783439:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 679 00000100:00100000:3.0:1788845020.783450:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641418496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.783485:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.783506:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79e6 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.783538:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.783549:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.783591:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.783609:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 16 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.783621:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660186368:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845020.783700:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845020.787848:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660186368:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845020.788171:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.788191:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.788194:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845020.788198:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.788209:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:2.0:1788845020.788283:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641418496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4834us (5121us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.788291:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 679 00000100:00100000:1.0:1788845020.789455:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.789563:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641418624 00000100:00100000:2.0:1788845020.789572:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641418624 00002000:00100000:2.0:1788845020.789626:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035929500 x1875739641418624/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:126/0 lens 16872/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.789637:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035929500 with x1875739641418624 00010000:00010000:2.0:1788845020.789645:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 17 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.789664:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 17 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845020.789673:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a035929500 00000100:00100000:2.0:1788845020.789719:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 17 00000100:00100000:2.0:1788845020.789729:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641418624:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845020.791478:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 811433 -> 811433 00000004:00010000:2.0:1788845020.791486:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845020 -> 1788845020 00000004:00010000:2.0:1788845020.791489:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845020 -> 1788845019 00000004:00010000:2.0:1788845020.791491:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845020 -> 1788845020 00000004:00010000:2.0:1788845020.791493:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:2.0:1788845020.792265:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641418624:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 2521us (2800us total) trans 4294967580 rc 0/0 00000100:00100000:2.0:1788845020.792276:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 17 00002000:00100000:2.0:1788845020.792289:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035929500 x1875739641418624/t4294967580(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:126/0 lens 16872/448 e 0 to 0 dl 1788845031 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845020.792299:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035929500 with x1875739641418624 00010000:00010000:2.0:1788845020.792309:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 19 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.792330:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 19 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845020.792338:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a035929500 00000100:00100000:1.0:1788845020.795646:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.795744:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641418752 00000100:00100000:0.0:1788845020.795753:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641418752 00010000:00010000:0.0:1788845020.795793:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 19 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.795894:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 246 00000100:00100000:0.0:1788845020.795905:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641418752:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845020.795918:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845020.795960:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 19 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.795978:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 19 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.796000:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287abfbd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79d1 expref: 19 pid: 5839 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.796016:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845020.796022:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.796035:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.796044:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.796108:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.796122:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d92680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660186496:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845020.796210:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.800544:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.800960:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d92680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660186496:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845020.800989:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845020.801044:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641418752:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5139us (5406us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845020.801065:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 246 00000100:00100000:3.0:1788845020.801139:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641418880 00000100:00100000:3.0:1788845020.801147:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641418880 00000100:00100000:3.0:1788845020.801201:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 393 00000100:00100000:3.0:1788845020.801211:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641418880:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845020.801474:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 811433 -> 811433 00000004:00010000:3.0:1788845020.801478:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845020 -> 1788845020 00000004:00010000:3.0:1788845020.801480:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:3.0:1788845020.801483:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845020 -> 1788845020 00000004:00010000:3.0:1788845020.801499:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:3.0:1788845020.801534:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641418880:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 326us (1000us total) trans 4294967581 rc 0/0 00000100:00100000:3.0:1788845020.801541:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 393 00000100:00100000:0.0:1788845020.805469:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.805514:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.805581:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641419136 00000100:00100000:3.0:1788845020.805591:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641419136 00000100:00100000:3.0:1788845020.805899:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 428 00000100:00100000:3.0:1788845020.805911:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641419136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.805940:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.805965:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287abfd9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79f4 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.806024:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02486a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660186624:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.806111:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.806127:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641419008 00000100:00100000:3.0:1788845020.806130:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641419008 00000100:00100000:3.0:1788845020.806159:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 680 00000100:00100000:3.0:1788845020.806166:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641419008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.806181:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.806193:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abfe0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79ed expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.806223:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.806246:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02485880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660186752:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.806287:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000004:00010000:2.0:1788845020.809864:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845019 -> 1788845020 00000100:00100000:2.0:1788845020.809927:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02485880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660186752:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845020.809969:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:2.0:1788845020.809990:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abfe0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79ed expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.810035:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.810039:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845020.810045:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287abfe0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79ed expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.810068:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:2.0:1788845020.810131:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641419008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3964us (4669us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845020.810141:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 680 00000100:00100000:3.0:1788845020.812128:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02486a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660186624:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.812161:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287abfd9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79f4 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.812184:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287abfd9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79f4 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.812196:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:3.0:1788845020.812418:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641419136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6508us (6902us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.812427:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 428 00000100:00100000:0.0:1788845020.816833:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.817123:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.817203:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641419264 00000100:00100000:2.0:1788845020.817213:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641419264 00000100:00100000:2.0:1788845020.817288:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 681 00000100:00100000:2.0:1788845020.817300:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641419264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.817321:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.817342:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfe7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a79fb expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.817381:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.817424:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660186880:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845020.817586:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.817813:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641419392 00000100:00100000:3.0:1788845020.817818:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641419392 00000100:00100000:3.0:1788845020.817922:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 429 00000100:00100000:3.0:1788845020.818046:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641419392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.818069:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.818082:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abfee lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a02 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.818120:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02486a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660187008:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.818154:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.821082:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660186880:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845020.821110:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:0.0:1788845020.821129:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfe7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79fb expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.821149:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845020.821152:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845020.821157:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287abfe7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79fb expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.821172:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f600, rc 0) 00000100:00100000:0.0:1788845020.821226:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641419264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3928us (4403us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845020.821234:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 681 00000100:00100000:1.0:1788845020.822655:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02486a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660187008:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845020.822688:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abfee lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a02 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.822706:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287abfee lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a02 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.822717:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845020.822785:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641419392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4846us (5662us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845020.822793:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 429 00000100:00100000:1.0:1788845020.825816:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.826089:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.826150:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641419520 00000100:00100000:3.0:1788845020.826159:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641419520 00000100:00100000:3.0:1788845020.826235:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 682 00000100:00100000:3.0:1788845020.826244:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641419520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.826288:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.826306:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287abff5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a09 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.826340:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.826378:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02487100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660187136:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.826440:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.828371:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641419648 00000100:00100000:3.0:1788845020.828383:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641419648 00000100:00100000:3.0:1788845020.828490:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 430 00000100:00100000:3.0:1788845020.828500:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641419648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.828516:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.828532:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abffc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a10 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.828592:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660187264:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.828660:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.830491:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02487100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660187136:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845020.830517:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:3.0:1788845020.830534:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287abff5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a09 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.830555:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845020.830558:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845020.830592:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287abff5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a09 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.830608:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:3.0:1788845020.830667:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641419520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4424us (4859us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.830676:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 682 00000100:00100000:3.0:1788845020.832531:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660187264:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.832565:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abffc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a10 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.832657:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287abffc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a10 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.832671:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845020.832728:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641419648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4228us (6638us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.832736:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 430 00000100:00100000:0.0:1788845020.835369:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.835424:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641419776 00000100:00100000:0.0:1788845020.835428:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.835433:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641419776 00000100:00100000:2.0:1788845020.835489:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 683 00000100:00100000:1.0:1788845020.835494:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641419904 00000100:00100000:1.0:1788845020.835498:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641419904 00000100:00100000:2.0:1788845020.835499:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641419776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.835526:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1788845020.835536:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 431 00000100:00100000:1.0:1788845020.835543:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641419904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.835546:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ac003 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a17 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.835551:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845020.835560:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a0be2217/0xb2187301287ac00a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a1e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.835608:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660187392:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845020.835689:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845020.839933:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660187392:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845020.839978:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a0be2217/0xb2187301287ac00a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a1e expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.840005:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a0be2217/0xb2187301287ac00a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a1e expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.840018:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a000, rc 0) 00000100:00100000:1.0:1788845020.840081:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641419904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4538us (4651us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845020.840107:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 431 00010000:00010000:2.0:1788845020.847699:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.848085:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660187520:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845020.848189:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.851505:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660187520:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845020.851534:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:3.0:1788845020.851553:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ac003 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a17 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.851646:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845020.851650:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845020.851656:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ac003 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a17 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.851687:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f800, rc 0) 00000100:00100000:3.0:1788845020.851746:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641419776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16248us (16385us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.851755:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 683 00000100:00100000:1.0:1788845020.855623:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.855710:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641420032 00000100:00100000:2.0:1788845020.855721:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641420032 00000100:00100000:2.0:1788845020.855781:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 684 00000100:00100000:2.0:1788845020.855791:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641420032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845020.855814:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845020.855835:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac011 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a25 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.855873:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.855951:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660187648:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845020.856037:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.860281:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.860490:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660187648:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845020.860516:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:2.0:1788845020.860535:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac011 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a25 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.860648:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.860651:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845020.860657:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac011 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a25 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.860675:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:2.0:1788845020.860848:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641420032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5056us (5232us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845020.860857:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 684 00000100:00100000:3.0:1788845020.860973:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641420160 00000100:00100000:3.0:1788845020.860980:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641420160 00000100:00100000:3.0:1788845020.861020:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 432 00000100:00100000:3.0:1788845020.861028:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641420160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.861042:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.861069:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac018 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a2c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.861120:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660187776:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.861168:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.864294:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660187776:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.864335:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac018 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a2c expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.864360:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac018 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a2c expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.864373:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845020.864433:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641420160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3405us (4161us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.864454:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 432 00000100:00100000:1.0:1788845020.869843:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.869923:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641420288 00000100:00100000:0.0:1788845020.869945:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641420288 00000100:00100000:0.0:1788845020.870003:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 685 00000100:00100000:0.0:1788845020.870013:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641420288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.870033:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.870063:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287ac01f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a33 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.870101:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.870140:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660187904:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845020.870219:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.872771:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.872908:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641420416 00000100:00100000:3.0:1788845020.872917:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641420416 00000100:00100000:3.0:1788845020.872986:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 433 00000100:00100000:3.0:1788845020.872995:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641420416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.873012:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.873030:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac026 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a3a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.873098:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660188032:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.873173:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.874815:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660187904:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845020.874844:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:3.0:1788845020.874862:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287ac01f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a33 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.875530:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845020.875535:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845020.875543:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0251a600/0xb2187301287ac01f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a33 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.875560:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a600, rc 0) 00000100:00100000:3.0:1788845020.875732:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641420288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5718us (5896us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.875741:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 685 00000100:00100000:3.0:1788845020.876600:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660188032:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.876645:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac026 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a3a expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.876667:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac026 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a3a expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.876679:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845020.876726:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641420416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3731us (3962us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.876733:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 433 00000100:00100000:0.0:1788845020.879247:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.879315:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641420544 00000100:00100000:3.0:1788845020.879324:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641420544 00000100:00100000:3.0:1788845020.879383:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 434 00000100:00100000:3.0:1788845020.879393:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641420544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.879412:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.879431:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a41 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.879462:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.879471:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(4 10) 00010000:00010000:3.0:1788845020.879476:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 10) 00010000:00010000:3.0:1788845020.879481:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1392640->1400831] (req 1392640->1400831) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.879535:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.879552:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 6 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.879566:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660188160:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845020.879688:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845020.882611:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788845020.882634:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660188288:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788845020.882700:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845020.882834:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660188288 00000100:00100000:0.0:1788845020.882841:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660188288 00000100:00100000:0.0:1788845020.882919:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 75 00000100:00100000:0.0:1788845020.882928:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660188288:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788845020.882974:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 85131264 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788845020.883018:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660188288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845020.883024:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 75 00000100:00100000:2.0:1788845020.883175:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660188288:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788845020.884487:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.884563:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641420672 00000100:00100000:0.0:1788845020.884570:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641420672 00010000:00010000:0.0:1788845020.884625:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 7 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.884666:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 247 00000100:00100000:0.0:1788845020.884676:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641420672:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845020.884689:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845020.884711:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 7 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.884730:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 7 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.884758:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 10) 00010000:00010000:0.0:1788845020.884763:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 10) 00010000:00010000:0.0:1788845020.884771:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.884782:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.884816:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.884830:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a0c700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660188416:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845020.884925:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00010000:00010000:3.0:1788845020.886198:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287abf3f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a797d expref: 7 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.886218:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660188160:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845020.886244:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.886727:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:3.0:1788845020.886826:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641420544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7433us (7585us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845020.886835:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 434 00000100:00100000:2.0:1788845020.890913:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a0c700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660188416:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845020.890959:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845020.891005:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641420672:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6330us (6521us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.891015:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 247 00000100:00100000:1.0:1788845020.893850:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.894374:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641420800 00000100:00100000:3.0:1788845020.894384:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641420800 00000100:00100000:3.0:1788845020.894446:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 686 00000100:00100000:3.0:1788845020.894456:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641420800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.894491:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.894510:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac034 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a48 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.894545:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.894763:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660188544:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.894852:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.897297:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660188544:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845020.897322:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:3.0:1788845020.897340:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac034 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a48 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.897363:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845020.897367:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845020.897372:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac034 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a48 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.897388:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845020.897450:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641420800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2994us (3609us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.897459:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 686 00000100:00100000:0.0:1788845020.900344:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.900408:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641420928 00000100:00100000:3.0:1788845020.900416:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641420928 00000100:00100000:3.0:1788845020.900464:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 687 00000100:00100000:3.0:1788845020.900471:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641420928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.900502:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.900520:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac03b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a4f expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.900549:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.900583:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660188672:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845020.900656:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.906438:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660188672:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845020.906466:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 811433 00010000:00010000:3.0:1788845020.906486:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac03b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a4f expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.906508:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845020.906511:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845020.906516:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac03b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a4f expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.906530:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:3.0:1788845020.906606:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641420928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6134us (6269us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.906615:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 687 00000100:00100000:1.0:1788845020.913322:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.913419:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641421056 00000100:00100000:0.0:1788845020.913430:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641421056 00000100:00100000:0.0:1788845020.913497:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 435 00000100:00100000:0.0:1788845020.913507:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641421056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.913527:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.913545:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287ac042 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a56 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.913610:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a0f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660188800:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845020.913697:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fe40 going to sleep for 11 seconds 00002000:00010000:1.0:1788845020.916006:0:7983:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1270398 -> 1397960 00000100:00100000:1.0:1788845020.916038:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a0f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660188800:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845020.916096:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287ac042 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a56 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.916122:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ade1d088/0xb2187301287ac042 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a56 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.916136:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251a200, rc 0) 00000100:00100000:1.0:1788845020.916194:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641421056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2688us (2885us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845020.916202:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 435 00000100:00100000:1.0:1788845020.919779:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.919841:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641421184 00000100:00100000:0.0:1788845020.919852:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641421184 00000100:00100000:0.0:1788845020.919926:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 436 00000100:00100000:0.0:1788845020.919945:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641421184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845020.919964:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845020.919983:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009f62ed95/0xb2187301287ac049 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a5d expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.920043:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034a0e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660188928:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845020.920176:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.922123:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034a0e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660188928:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845020.922162:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009f62ed95/0xb2187301287ac049 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a5d expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.922211:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009f62ed95/0xb2187301287ac049 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a5d expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.922225:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0251be00, rc 0) 00000100:00100000:3.0:1788845020.922359:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641421184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2424us (2589us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845020.922369:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 436 00000100:00100000:1.0:1788845020.924500:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.924601:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641421312 00000100:00100000:3.0:1788845020.924610:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641421312 00000100:00100000:3.0:1788845020.924695:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 437 00000100:00100000:3.0:1788845020.924706:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641421312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.924723:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.924741:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a64 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.924773:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.924782:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 10) 00010000:00010000:3.0:1788845020.924787:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 10) 00010000:00010000:3.0:1788845020.924793:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1089536->1155071] (req 1089536->1155071) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.924918:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.925096:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 7 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.925110:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660189056:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845020.925311:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.928158:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.928307:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641421440 00000100:00100000:1.0:1788845020.928316:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641421440 00002000:00100000:1.0:1788845020.928364:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1392640->1397959) req@ffff949f02487100 x1875739641421440/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:126/0 lens 5808/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.928376:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02487100 with x1875739641421440 ext(1392640->1397959) 00010000:00010000:1.0:1788845020.928397:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 8 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.928418:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 8 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845020.928427:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f02487100 00000100:00100000:1.0:1788845020.928495:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 8 00000100:00100000:1.0:1788845020.928505:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02487100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641421440:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845020.929108:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02487100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641421440:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 603us (959us total) trans 4294967378 rc 0/0 00000100:00100000:1.0:1788845020.929116:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 8 00002000:00100000:1.0:1788845020.929124:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1392640->1397959) req@ffff949f02487100 x1875739641421440/t4294967378(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:126/0 lens 5808/448 e 0 to 0 dl 1788845031 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.929133:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02487100 with x1875739641421440 ext(1392640->1397959) 00010000:00010000:1.0:1788845020.929139:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845020.929154:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845020.929161:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f02487100 00000100:00100000:2.0:1788845020.929254:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660189056:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845020.929285:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1089536->1155071] (req 1089536->1155071) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.929301:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:2.0:1788845020.929330:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641421312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4628us (4842us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.929337:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 437 00000100:00100000:0.0:1788845020.931746:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.931838:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641421568 00000100:00100000:2.0:1788845020.931846:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641421568 00010000:00010000:2.0:1788845020.931907:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.931949:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 248 00000100:00100000:2.0:1788845020.931960:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641421568:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845020.931974:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845020.931995:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.932013:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.932036:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac02d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1400831) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a41 expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.932060:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 10) 00010000:00010000:2.0:1788845020.932065:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 10) 00010000:00010000:2.0:1788845020.932072:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.932083:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.932138:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.932151:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660189184:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845020.932231:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845020.935291:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660189184:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845020.935322:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845020.935371:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641421568:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3412us (3633us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845020.935382:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 248 00000100:00100000:3.0:1788845020.942946:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845020.943354:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641421696 00000100:00100000:1.0:1788845020.943364:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641421696 00002000:00100000:1.0:1788845020.943416:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1400831) req@ffff949f02485180 x1875739641421696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.943440:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02485180 with x1875739641421696 ext(1048576->1400831) 00002000:00010000:1.0:1788845020.943446:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02485180 00000100:00100000:1.0:1788845020.943484:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 253 00000100:00100000:1.0:1788845020.943494:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02485180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641421696:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845020.949614:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.949727:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641421824 00000100:00100000:3.0:1788845020.949738:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641421824 00002000:00100000:3.0:1788845020.949785:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034a0f480 x1875739641421824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/0 e 0 to 0 dl 1788845031 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.949797:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034a0f480 with x1875739641421824 00000100:00100000:3.0:1788845020.949837:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 394 00000100:00100000:3.0:1788845020.949847:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641421824:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845020.952290:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02485180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641421696:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8786us (9396us total) trans 0 rc 352256/352256 00000100:00100000:1.0:1788845020.952304:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 253 00002000:00100000:1.0:1788845020.952318:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1400831) req@ffff949f02485180 x1875739641421696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/440 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 352256/352256 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845020.952328:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02485180 with x1875739641421696 ext(1048576->1400831) 00002000:00010000:1.0:1788845020.952334:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02485180 00000100:00100000:3.0:1788845020.966458:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641421824:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 16601us (16843us total) trans 0 rc 815104/815104 00000100:00100000:3.0:1788845020.966474:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 394 00002000:00100000:3.0:1788845020.966489:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034a0f480 x1875739641421824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:126/0 lens 488/440 e 0 to 0 dl 1788845031 ref 1 fl Complete:/600/0 rc 815104/815104 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845020.966501:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034a0f480 with x1875739641421824 00000100:00100000:0.0:1788845020.971538:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.971683:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641421952 00000100:00100000:3.0:1788845020.971693:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641421952 00000100:00100000:3.0:1788845020.971772:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 395 00000100:00100000:3.0:1788845020.971782:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641421952:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845020.972220:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 811433 -> 811433 00000004:00010000:3.0:1788845020.972224:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845020 -> 1788845020 00000004:00010000:3.0:1788845020.972227:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845020 -> 1788845019 00000004:00010000:3.0:1788845020.972229:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845020 -> 1788845020 00000004:00010000:3.0:1788845020.972232:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:3.0:1788845020.972295:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641421952:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 514us (766us total) trans 4294967582 rc 0/0 00000100:00100000:3.0:1788845020.972303:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 395 00000100:00100000:0.0:1788845020.976604:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845020.976745:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641422208 00000100:00100000:2.0:1788845020.976755:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641422208 00000100:00100000:2.0:1788845020.976812:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 688 00000100:00100000:2.0:1788845020.976822:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641422208:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845020.976879:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845020.976944:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287abf4d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7984 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.976964:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845020.976968:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845020.977199:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.977214:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.977242:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.977277:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.977295:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 17 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.977310:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660189312:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845020.977386:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845020.980647:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.980746:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641422336 00000100:00100000:3.0:1788845020.980756:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641422336 00010000:00010000:3.0:1788845020.980795:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 18 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.980836:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 249 00000100:00100000:3.0:1788845020.980845:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641422336:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845020.980860:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845020.980867:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 18 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.980929:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 18 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.980964:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845020.980970:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.980993:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.981002:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.981022:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845020.981082:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641422336:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 237us (446us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845020.981090:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 249 00010000:00010000:2.0:1788845020.981718:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287abf5b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a798b expref: 17 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.981737:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660189312:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845020.981763:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.981773:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845020.981779:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845020.981958:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967583 00000004:00010000:2.0:1788845020.982010:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 811433 -> 811433 00000004:00010000:2.0:1788845020.982013:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845020 -> 1788845021 00000004:00010000:2.0:1788845020.982025:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:2.0:1788845020.982028:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845020 -> 1788845021 00000004:00010000:2.0:1788845020.982030:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000004:00010000:2.0:1788845020.982037:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001f5d826f reply state 00000000bb155a30 transno 4294967584 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845020.982046:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845020.982140:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641422208:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5319us (5546us total) trans 4294967584 rc 0/0 00000100:00100000:2.0:1788845020.982147:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 688 00000020:00080000:2.0:1788845020.982152:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845020.989703:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.989723:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.989731:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.989737:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031d3f400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845020.989742:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.989750:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.989768:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845020.989772:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845020.989776:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.989803:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ac057 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845020.990446:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845020.990507:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641422464 00000100:00100000:3.0:1788845020.990517:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641422464 00000100:00100000:3.0:1788845020.990577:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 438 00000100:00100000:3.0:1788845020.990588:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641422464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845020.990607:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845020.990626:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a6b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.990657:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.990667:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 10) 00010000:00010000:3.0:1788845020.990671:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 10) 00010000:00010000:3.0:1788845020.990677:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.990721:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.990739:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 8 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845020.990754:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02487480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660189440:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845020.990850:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845020.993768:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02487480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660189440:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845020.993804:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845020.993825:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845020.993967:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641422464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3380us (3527us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845020.993976:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 438 00000100:00100000:1.0:1788845020.998434:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845020.998702:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641422592 00000100:00100000:0.0:1788845020.998714:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641422592 00010000:00010000:0.0:1788845020.998757:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 8 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.998851:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 250 00000100:00100000:0.0:1788845020.998863:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008914850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641422592:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845020.998880:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845020.998948:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 8 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.998967:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 8 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.998991:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ac050 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1089536->1155071) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a64 expref: 8 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.999027:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 10) 00010000:00010000:0.0:1788845020.999032:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 10) 00010000:00010000:0.0:1788845020.999041:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.999084:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845020.999130:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845020.999165:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660189568:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845020.999250:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.002204:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660189568:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845021.002233:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845021.002294:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008914850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641422592:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3433us (3870us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.002305:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 250 00000100:00100000:3.0:1788845021.003916:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.003960:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.004038:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641422848 00000100:00100000:1.0:1788845021.004048:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641422848 00002000:00010000:1.0:1788845021.004108:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845021.004113:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02487800 with x1875739641422848 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845021.004125:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02487800. 00000100:00100000:1.0:1788845021.004154:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 254 00000100:00100000:1.0:1788845021.004163:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02487800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641422848:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845021.004491:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1397960 -> 0 00002000:00010000:1.0:1788845021.004495:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845020 -> 1788845021 00002000:00010000:1.0:1788845021.004498:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845020 -> 1788845020 00002000:00010000:1.0:1788845021.004511:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845020 -> 1788845021 00002000:00010000:1.0:1788845021.004513:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 341 -> 341 00000100:00100000:1.0:1788845021.004566:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02487800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641422848:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 403us (604us total) trans 4294967379 rc 0/0 00000100:00100000:1.0:1788845021.004573:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 254 00002000:00010000:1.0:1788845021.004577:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845021.004580:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02487800 with x1875739641422848 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845021.004584:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02487800. 00000100:00100000:2.0:1788845021.004874:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641422720 00000100:00100000:2.0:1788845021.004879:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641422720 00002000:00010000:2.0:1788845021.004935:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845021.004940:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02487480 with x1875739641422720 00000100:00100000:2.0:1788845021.004966:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 396 00000100:00100000:2.0:1788845021.004973:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02487480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641422720:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:2.0:1788845021.005300:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 811433 -> 31721 00000004:00010000:2.0:1788845021.005304:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845021.005307:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:2.0:1788845021.005309:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845021.005311:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:2.0:1788845021.005351:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02487480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641422720:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 378us (1470us total) trans 4294967585 rc 0/0 00000100:00100000:2.0:1788845021.005358:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 396 00002000:00010000:2.0:1788845021.005362:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845021.005364:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02487480 with x1875739641422720 00000100:00080000:3.0:1788845021.005739:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788845021.005745:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845021.011121:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.011284:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641422976 00000100:00100000:0.0:1788845021.011295:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641422976 00000100:00100000:0.0:1788845021.011359:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 689 00000100:00100000:0.0:1788845021.011369:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02484380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641422976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.011397:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.011421:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50000/0xb2187301287ac065 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a72 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.011512:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.011526:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.011538:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845021.011567:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 31721 00000004:00200000:0.0:1788845021.011571:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845021.011778:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.011803:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50000/0xb2187301287ac065 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.011820:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.011829:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.011832:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845021.011835:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a50400, rc 0) 00000100:00100000:0.0:1788845021.012068:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02484380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641422976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 698us (955us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.012076:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 689 00000100:00100000:1.0:1788845021.016224:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.016349:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641423104 00000100:00100000:3.0:1788845021.016360:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641423104 00000100:00100000:3.0:1788845021.016467:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 690 00000100:00100000:3.0:1788845021.016479:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641423104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.016500:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.016520:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac073 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a79 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.016620:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac07a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.016636:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac07a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.016667:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac07a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845021.016696:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 31721 00000004:00200000:3.0:1788845021.016700:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845021.016719:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac07a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.016747:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac073 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a79 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.016764:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac07a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a79 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.016773:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.016776:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845021.016779:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:3.0:1788845021.016844:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641423104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 367us (630us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.016850:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 690 00000100:00080000:0.0:1788845021.018171:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845021.018954:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.019084:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641423232 00000100:00100000:3.0:1788845021.019093:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641423232 00000100:00100000:3.0:1788845021.019149:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 691 00000100:00100000:3.0:1788845021.019159:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641423232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.019178:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.019197:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac081 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a80 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.019251:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.019297:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660189696:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.019378:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.021123:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000004:00010000:3.0:1788845021.022922:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845019 -> 1788845021 00000100:00100000:3.0:1788845021.022973:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660189696:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845021.023004:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 31721 00010000:00010000:3.0:1788845021.023029:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac081 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a80 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.023060:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.023063:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.023069:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac081 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a80 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.023086:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:3.0:1788845021.023150:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641423232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3992us (4200us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.023158:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 691 00000100:00100000:0.0:1788845021.024141:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641423360 00000100:00100000:0.0:1788845021.024153:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641423360 00000100:00100000:1.0:1788845021.024286:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 439 00000100:00100000:1.0:1788845021.024296:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641423360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.024313:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.024331:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac088 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a87 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.024387:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660189824:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.024462:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00080000:0.0:1788845021.025823:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1788845021.026643:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845020 -> 1788845021 00000100:00100000:2.0:1788845021.026675:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660189824:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845021.026718:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac088 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a87 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.026743:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac088 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a87 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.026757:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a800, rc 0) 00000100:00100000:2.0:1788845021.026858:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641423360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2562us (5740us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.026867:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 439 00000100:00100000:1.0:1788845021.029308:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.029356:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.029452:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641423616 00000100:00100000:0.0:1788845021.029464:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641423616 00000100:00100000:0.0:1788845021.029530:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 440 00000100:00100000:0.0:1788845021.029540:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641423616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.029557:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.029574:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005845d26a/0xb2187301287ac08f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a95 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.029632:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660189952:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.029721:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.030185:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641423488 00000100:00100000:3.0:1788845021.030190:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641423488 00000100:00100000:3.0:1788845021.030234:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 692 00000100:00100000:3.0:1788845021.030242:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641423488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.030253:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.030273:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac096 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a8e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.030303:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.030329:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660190080:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.030376:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.033825:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660189952:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845021.033869:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005845d26a/0xb2187301287ac08f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a95 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.033944:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005845d26a/0xb2187301287ac08f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a95 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.033960:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a50000, rc 0) 00000100:00100000:0.0:1788845021.034020:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641423616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4481us (4662us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.034028:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 440 00000100:00100000:3.0:1788845021.034475:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660190080:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845021.034491:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 31721 00010000:00010000:3.0:1788845021.034504:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac096 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a8e expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.034518:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.034521:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.034526:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac096 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a8e expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.034538:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:3.0:1788845021.034567:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641423488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4327us (5270us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.034574:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 692 00000100:00100000:1.0:1788845021.038496:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.038548:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.038622:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641423744 00000100:00100000:0.0:1788845021.038632:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641423744 00000100:00100000:0.0:1788845021.038703:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 693 00000100:00100000:0.0:1788845021.038714:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641423744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.038735:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.038756:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50200/0xb2187301287ac09d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7a9c expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.038792:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.038833:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660190208:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.038952:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.039132:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641423872 00000100:00100000:3.0:1788845021.039137:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641423872 00000100:00100000:3.0:1788845021.039172:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 441 00000100:00100000:3.0:1788845021.039180:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641423872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.039189:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.039214:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ac0a4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7aa3 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.039249:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660190336:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.039289:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.042660:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660190208:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845021.042709:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 31721 00010000:00010000:0.0:1788845021.042730:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50200/0xb2187301287ac09d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a9c expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.042753:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.042757:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845021.042761:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50200/0xb2187301287ac09d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a9c expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.042776:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a50200, rc 0) 00000100:00100000:0.0:1788845021.042834:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641423744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4121us (4349us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.042842:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 693 00000100:00100000:3.0:1788845021.043343:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660190336:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845021.043365:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ac0a4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7aa3 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.043383:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ac0a4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7aa3 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.043393:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:3.0:1788845021.043432:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641423872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4252us (4883us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.043439:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 441 00000100:00100000:2.0:1788845021.047365:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.047447:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.047518:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641424128 00000100:00100000:3.0:1788845021.047531:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641424128 00000100:00100000:3.0:1788845021.047603:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 442 00000100:00100000:3.0:1788845021.047615:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641424128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.047636:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.047656:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ac0ab lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ab1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.047718:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660190464:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.047826:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.049980:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660190464:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845021.050020:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ac0ab lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ab1 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.050046:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ac0ab lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ab1 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.050071:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:2.0:1788845021.050131:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641424128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2518us (2683us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.050140:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 442 00000100:00100000:1.0:1788845021.052200:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641424000 00000100:00100000:1.0:1788845021.052212:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641424000 00000100:00100000:1.0:1788845021.052281:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 694 00000100:00100000:1.0:1788845021.052292:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641424000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.052330:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.052353:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac0b2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7aaa expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.052387:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.052425:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660190592:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.052518:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.054771:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660190592:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845021.054804:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 31721 00010000:00010000:1.0:1788845021.054824:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac0b2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7aaa expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.054844:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.054847:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.054852:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac0b2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7aaa expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.054868:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b600, rc 0) 00000100:00100000:1.0:1788845021.055357:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641424000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3064us (8104us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.055366:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 694 00000100:00100000:0.0:1788845021.062300:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.062348:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.062442:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641424256 00000100:00100000:2.0:1788845021.062453:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641424256 00000100:00100000:2.0:1788845021.062512:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 695 00000100:00100000:2.0:1788845021.062522:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641424256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.062544:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.062564:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287ac0b9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ab8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.062601:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.062640:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660190720:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.062721:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.062802:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641424384 00000100:00100000:1.0:1788845021.062807:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641424384 00000100:00100000:1.0:1788845021.062841:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 443 00000100:00100000:1.0:1788845021.062848:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641424384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.062868:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.062879:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e41520c/0xb2187301287ac0c0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7abf expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.062953:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660190848:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.062985:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.064953:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660190720:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845021.064983:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 31721 00010000:00010000:2.0:1788845021.065001:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287ac0b9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ab8 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.065022:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.065025:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845021.065030:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287ac0b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ab8 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.065044:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:2.0:1788845021.065103:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641424256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2582us (2814us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.065111:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 695 00000100:00100000:1.0:1788845021.067197:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660190848:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845021.067238:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e41520c/0xb2187301287ac0c0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7abf expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.067262:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e41520c/0xb2187301287ac0c0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7abf expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.067659:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b400, rc 0) 00000100:00100000:1.0:1788845021.067757:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641424384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4908us (5407us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.067767:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 443 00000100:00100000:0.0:1788845021.071390:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.071431:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.071779:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641424512 00000100:00100000:3.0:1788845021.071789:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641424512 00000100:00100000:3.0:1788845021.071850:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 696 00000100:00100000:3.0:1788845021.071860:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641424512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.071878:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.072384:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51200/0xb2187301287ac0c7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ac6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.072424:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.072465:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660190976:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.072547:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.073179:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641424640 00000100:00100000:1.0:1788845021.073186:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641424640 00000100:00100000:1.0:1788845021.073229:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 444 00000100:00100000:1.0:1788845021.073237:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641424640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.073252:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.073266:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287ac0ce lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7acd expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.073328:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660191104:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.073370:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.077300:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660191104:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845021.077345:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287ac0ce lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7acd expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.077371:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287ac0ce lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7acd expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.077387:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b600, rc 0) 00000100:00100000:1.0:1788845021.077447:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641424640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4209us (6014us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.077456:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 444 00000100:00100000:0.0:1788845021.077950:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660190976:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845021.077968:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 31721 00010000:00010000:0.0:1788845021.077996:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51200/0xb2187301287ac0c7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ac6 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.078011:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.078014:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845021.078018:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51200/0xb2187301287ac0c7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ac6 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.078029:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51200, rc 0) 00000100:00100000:0.0:1788845021.078064:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641424512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6206us (6685us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.078071:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 696 00000100:00100000:3.0:1788845021.081948:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.082036:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641424896 00000100:00100000:0.0:1788845021.082045:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641424896 00000100:00100000:0.0:1788845021.082115:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 697 00000100:00100000:0.0:1788845021.082124:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02486a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641424896:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845021.082177:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845021.082202:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac07a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a79 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.082220:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.082224:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845021.082431:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.082446:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.082472:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.082502:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.082520:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 15 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.082536:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660191232:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845021.082691:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.085170:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.085306:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660191232:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845021.085335:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.089128:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641425024 00000100:00100000:3.0:1788845021.089142:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641425024 00010000:00010000:0.0:1788845021.093857:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 16 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.094537:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 251 00000100:00100000:0.0:1788845021.094551:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641425024:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845021.094580:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845021.094587:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 16 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.094605:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 16 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.094641:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a50400/0xb2187301287ac06c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a72 expref: 16 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.094657:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.094661:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.094672:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.094680:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.094715:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845021.094778:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641425024:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 231us (9615us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.094785:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 251 00010000:00010000:3.0:1788845021.094853:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.094862:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845021.095013:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967586 00000004:00010000:3.0:1788845021.095075:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 31721 -> 31721 00000004:00010000:3.0:1788845021.095079:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.095082:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845019 00000004:00010000:3.0:1788845021.095084:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.095086:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000004:00010000:3.0:1788845021.095092:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006e9e2b2e reply state 00000000f889bbd2 transno 4294967587 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.095101:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845021.095168:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02486a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641424896:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13045us (13229us total) trans 4294967587 rc 0/0 00000100:00100000:3.0:1788845021.095176:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 697 00000020:00080000:3.0:1788845021.095179:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845021.100834:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.100853:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.100861:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.100868:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02a51a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845021.100872:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.100947:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.100966:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.100969:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845021.100974:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.100980:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51a00/0xb2187301287ac0d5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.101218:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.101737:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641425152 00000100:00100000:1.0:1788845021.101747:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641425152 00000100:00100000:1.0:1788845021.101811:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 698 00000100:00100000:1.0:1788845021.101822:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02485880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641425152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.101849:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.101870:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ad4 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.101935:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.101947:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.101983:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.102018:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 14 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.102033:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660191360:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845021.102125:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.107443:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.107517:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641425280 00000100:00100000:3.0:1788845021.107527:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641425280 00010000:00010000:3.0:1788845021.107691:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.107738:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 252 00000100:00100000:3.0:1788845021.107748:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641425280:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845021.107761:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845021.107786:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.107803:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.107825:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.107831:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.107844:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.107853:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.107912:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.108062:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287abfd2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a79e6 expref: 15 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.108080:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660191360:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845021.108109:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.108121:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.108124:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845021.108127:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b400, rc 0) 00000100:00100000:2.0:1788845021.108181:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02485880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641425152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6360us (6966us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.108189:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 698 00000100:00100000:3.0:1788845021.110779:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660191488:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845021.110872:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.117212:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.117383:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660191488:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845021.117407:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845021.117467:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641425280:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9720us (10035us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.117478:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 252 00000100:00100000:0.0:1788845021.117580:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641425408 00000100:00100000:0.0:1788845021.117587:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641425408 00000100:00100000:0.0:1788845021.117635:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 445 00000100:00100000:0.0:1788845021.117644:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641425408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.117662:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.117680:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7adb expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.117712:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.117740:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 10) 00010000:00010000:0.0:1788845021.117744:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 10) 00010000:00010000:0.0:1788845021.117750:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.117779:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.117797:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 7 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.117811:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660191616:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845021.117859:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.122158:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.122262:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641425536 00000100:00100000:3.0:1788845021.122272:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641425536 00010000:00010000:3.0:1788845021.122311:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 8 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.122355:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 253 00000100:00100000:3.0:1788845021.122365:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641425536:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845021.122379:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845021.122401:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 8 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.122420:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 8 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.122445:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 10) 00010000:00010000:3.0:1788845021.122450:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 10) 00010000:00010000:3.0:1788845021.122459:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.122483:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.122521:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.122536:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660191744:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845021.122633:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788845021.122734:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac05e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7a6b expref: 8 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.122750:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660191616:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845021.122779:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.122788:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a50400, rc 0) 00000100:00100000:0.0:1788845021.122817:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641425408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5173us (5615us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.122833:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 445 00000100:00100000:3.0:1788845021.128298:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660191744:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845021.128324:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845021.128379:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641425536:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6014us (6230us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.128389:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 253 00000100:00100000:1.0:1788845021.130976:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.131169:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641425664 00000100:00100000:2.0:1788845021.131179:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641425664 00002000:00010000:2.0:1788845021.131220:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1253382->18446744073709551615) 00002000:00100000:2.0:1788845021.131226:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f1880 with x1875739641425664 ext(1253382->18446744073709551615) 00002000:00010000:2.0:1788845021.131238:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f1880. 00000100:00100000:2.0:1788845021.131268:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 255 00000100:00100000:2.0:1788845021.131279:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641425664:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845021.131502:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1253382 00002000:00010000:2.0:1788845021.131506:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00002000:00010000:2.0:1788845021.131508:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845021 -> 1788845020 00002000:00010000:2.0:1788845021.131511:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00002000:00010000:2.0:1788845021.131527:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 341 -> 341 00000100:00100000:2.0:1788845021.132771:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641425664:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 1486us (1796us total) trans 4294967380 rc 0/0 00000100:00100000:2.0:1788845021.132782:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 255 00002000:00010000:2.0:1788845021.132787:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1253382->18446744073709551615) 00002000:00100000:2.0:1788845021.132791:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f1880 with x1875739641425664 ext(1253382->18446744073709551615) 00002000:00010000:2.0:1788845021.132798:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f1880. 00000100:00100000:2.0:1788845021.136986:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.137087:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641425792 00000100:00100000:3.0:1788845021.137098:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641425792 00000100:00100000:3.0:1788845021.137197:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 699 00000100:00100000:3.0:1788845021.137207:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641425792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.137227:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.137247:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0ea lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ae2 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.137332:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac0f1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.137366:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac0f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.137379:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac0f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845021.137408:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1253382 00000004:00200000:3.0:1788845021.137412:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845021.137431:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac0f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.137458:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0ea lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae2 expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.137476:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac0f1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae2 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.137485:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.137488:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845021.137492:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:3.0:1788845021.137556:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641425792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 350us (672us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.137564:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 699 00000100:00100000:3.0:1788845021.141475:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.141526:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641425920 00000100:00100000:1.0:1788845021.141536:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641425920 00002000:00100000:1.0:1788845021.141743:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02484a80 x1875739641425920/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.141755:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02484a80 with x1875739641425920 00000100:00100000:1.0:1788845021.141790:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 397 00000100:00100000:1.0:1788845021.141800:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02484a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641425920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.142117:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02484a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641425920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 317us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845021.142125:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 397 00002000:00100000:1.0:1788845021.142131:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02484a80 x1875739641425920/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.142138:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02484a80 with x1875739641425920 00000100:00100000:1.0:1788845021.147129:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.147218:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641426048 00000100:00100000:2.0:1788845021.147229:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641426048 00002000:00100000:2.0:1788845021.147288:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f024d5f80 x1875739641426048/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.147301:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f024d5f80 with x1875739641426048 00000100:00100000:2.0:1788845021.147343:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 398 00000100:00100000:2.0:1788845021.147353:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641426048:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845021.147595:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641426048:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.147601:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 398 00002000:00100000:2.0:1788845021.147607:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f024d5f80 x1875739641426048/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.147614:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f024d5f80 with x1875739641426048 00000100:00100000:2.0:1788845021.155214:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.155364:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641426176 00000100:00100000:3.0:1788845021.155374:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641426176 00002000:00100000:3.0:1788845021.155420:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48e00 x1875739641426176/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 504/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.155452:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48e00 with x1875739641426176 00000100:00100000:3.0:1788845021.155488:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 399 00000100:00100000:3.0:1788845021.155498:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641426176:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.162938:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641426176:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 7420us (7711us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.162955:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 399 00002000:00100000:3.0:1788845021.162969:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48e00 x1875739641426176/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 504/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.162980:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48e00 with x1875739641426176 00000100:00100000:2.0:1788845021.166180:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.166289:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641426304 00000100:00100000:1.0:1788845021.166299:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641426304 00002000:00100000:1.0:1788845021.166358:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01ced180 x1875739641426304/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 12776/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.166370:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01ced180 with x1875739641426304 00000100:00100000:1.0:1788845021.166450:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 400 00000100:00100000:1.0:1788845021.166461:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641426304:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845021.166853:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 31721 -> 1015808 00000004:00010000:1.0:1788845021.166858:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:1.0:1788845021.166860:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:1.0:1788845021.166862:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:1.0:1788845021.166864:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:1.0:1788845021.166968:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641426304:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 508us (796us total) trans 4294967588 rc 0/0 00000100:00100000:1.0:1788845021.166976:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 400 00002000:00100000:1.0:1788845021.166984:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01ced180 x1875739641426304/t4294967588(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 12776/448 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.166992:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01ced180 with x1875739641426304 00000100:00100000:2.0:1788845021.169314:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.169377:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641426432 00000100:00100000:3.0:1788845021.169387:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641426432 00000100:00100000:3.0:1788845021.169435:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 700 00000100:00100000:3.0:1788845021.169443:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641426432:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845021.200791:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967586 is committed 00000001:00080000:1.0:1788845021.200812:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967588 is committed 00000001:00080000:1.0:1788845021.200824:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967585 is committed 00000100:00100000:2.0:1788845021.201220:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641426432:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 31771us (31908us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.201248:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 700 00000100:00100000:0.0:1788845021.205739:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.205829:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641426560 00000100:00100000:3.0:1788845021.205839:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641426560 00000100:00100000:3.0:1788845021.205963:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 701 00000100:00100000:3.0:1788845021.205975:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641426560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.205997:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.206017:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ae9 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.206058:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.206086:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.206122:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.206140:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 14 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.206155:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660191872:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.206241:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.209657:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660191872:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845021.209698:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.209718:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.209721:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.209727:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.209738:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:3.0:1788845021.209789:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641426560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3816us (4058us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.209799:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 701 00000100:00100000:1.0:1788845021.209976:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.210006:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641426688 00000100:00100000:1.0:1788845021.210012:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641426688 00010000:00010000:1.0:1788845021.210057:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 15 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.210088:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 254 00000100:00100000:1.0:1788845021.210109:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641426688:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845021.210124:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845021.210150:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 15 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.210166:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 15 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.210188:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac0dc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ad4 expref: 15 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.210203:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.210207:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.210216:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.210225:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.210257:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.210271:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f024d5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660192000:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845021.210320:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.214317:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f024d5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660192000:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845021.214349:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845021.214405:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641426688:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4297us (4429us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.214416:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 254 00000100:00100000:0.0:1788845021.220304:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.220403:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641426816 00000100:00100000:1.0:1788845021.220425:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641426816 00002000:00100000:1.0:1788845021.220477:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb5880 x1875739641426816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.220490:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb5880 with x1875739641426816 00000100:00100000:1.0:1788845021.220528:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 401 00000100:00100000:1.0:1788845021.220539:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641426816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.232976:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641426816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12427us (12670us total) trans 0 rc 1015808/1015808 00000100:00100000:1.0:1788845021.232992:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 401 00002000:00100000:1.0:1788845021.233007:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a007eb5880 x1875739641426816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.233019:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a007eb5880 with x1875739641426816 00000100:00100000:1.0:1788845021.237019:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.237153:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641426944 00000100:00100000:2.0:1788845021.237164:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641426944 00000100:00100000:2.0:1788845021.237223:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 702 00000100:00100000:2.0:1788845021.237233:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641426944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.237253:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.237274:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac0ff lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7af0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.237310:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac0ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af0 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845021.237323:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1015808 00010000:00010000:2.0:1788845021.237331:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac0ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af0 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.237345:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.237348:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845021.237352:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864e00, rc 0) 00000100:00100000:2.0:1788845021.237447:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641426944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 215us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.237470:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 702 00000100:00100000:2.0:1788845021.240959:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.241155:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641427072 00000100:00100000:3.0:1788845021.241177:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641427072 00000100:00100000:3.0:1788845021.241231:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 703 00000100:00100000:3.0:1788845021.241240:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cedc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641427072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.241266:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.241269:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845021.241307:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac0ff lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af0 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.241325:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.241329:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845021.241342:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7af7 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.241365:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.241374:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.241401:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.241416:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 9 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.241430:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660192128:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.241510:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.244495:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660192128:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845021.244529:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.244547:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.244550:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.244555:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.244577:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845021.244633:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641427072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3393us (3686us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.244642:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 703 00000100:00100000:1.0:1788845021.249355:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.249466:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641427200 00000100:00100000:2.0:1788845021.249476:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641427200 00010000:00010000:2.0:1788845021.249518:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 10 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.249558:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 255 00000100:00100000:2.0:1788845021.249723:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641427200:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845021.249741:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845021.249772:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 10 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.249789:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 10 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.249814:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac0f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae9 expref: 10 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.249832:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.249838:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.249850:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.249877:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.250144:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.250159:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ced500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660192256:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845021.250460:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.253961:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.254032:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641427328 00000100:00100000:3.0:1788845021.254041:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641427328 00000100:00100000:3.0:1788845021.254115:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 402 00000100:00100000:3.0:1788845021.254125:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641427328:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845021.254446:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1015808 -> 1015808 00000004:00010000:3.0:1788845021.254451:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.254453:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:3.0:1788845021.254456:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.254458:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 134 00000100:00100000:3.0:1788845021.254530:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641427328:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 405us (578us total) trans 4294967589 rc 0/0 00000100:00100000:3.0:1788845021.254538:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 402 00000100:00100000:0.0:1788845021.255356:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ced500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660192256:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845021.255382:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845021.255412:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641427200:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5845us (6069us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.255422:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 255 00000100:00100000:0.0:1788845021.260450:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.260844:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641427456 00000100:00100000:3.0:1788845021.260869:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641427456 00000100:00100000:3.0:1788845021.260974:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 403 00000100:00100000:3.0:1788845021.260985:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641427456:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845021.261265:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1015808 -> 1015808 00000004:00010000:3.0:1788845021.261269:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.261272:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845019 -> 1788845019 00000004:00010000:3.0:1788845021.261274:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.261276:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 134 -> 134 00000100:00100000:3.0:1788845021.261351:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641427456:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 368us (915us total) trans 4294967590 rc 0/0 00000100:00100000:3.0:1788845021.261359:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 403 00000100:00100000:2.0:1788845021.264257:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.264359:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641427584 00000100:00100000:1.0:1788845021.264369:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641427584 00000100:00100000:1.0:1788845021.264429:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 704 00000100:00100000:1.0:1788845021.264440:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641427584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.264461:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.264484:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac10d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7afe expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.264585:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.264600:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.264625:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845021.264653:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1253382 00000004:00200000:1.0:1788845021.264658:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845021.264677:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.264706:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac10d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.264725:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.264734:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.264736:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845021.264740:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a800, rc 0) 00000100:00100000:1.0:1788845021.264807:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641427584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 368us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845021.264814:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 704 00000100:00100000:2.0:1788845021.268001:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.268033:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.268122:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641427840 00000100:00100000:3.0:1788845021.268132:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641427840 00000100:00100000:3.0:1788845021.268182:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 446 00000100:00100000:3.0:1788845021.268192:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641427840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.268212:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.268230:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac11b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b0c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.268286:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660192384:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.268365:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.268431:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641427712 00000100:00100000:0.0:1788845021.268436:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641427712 00000100:00100000:0.0:1788845021.268481:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 705 00000100:00100000:0.0:1788845021.268489:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cefb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641427712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.268499:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.268511:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51e00/0xb2187301287ac122 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b05 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.268542:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.268577:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660192512:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.268605:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00002000:00010000:3.0:1788845021.272075:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845020 -> 1788845021 00000100:00100000:3.0:1788845021.272111:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660192384:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845021.272153:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac11b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b0c expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.272205:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac11b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b0c expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.272218:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:3.0:1788845021.272277:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641427840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4085us (4241us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.272285:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 446 00000004:00010000:0.0:1788845021.272382:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845019 -> 1788845021 00000100:00100000:0.0:1788845021.272403:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660192512:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845021.272421:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1015808 00010000:00010000:0.0:1788845021.272443:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51e00/0xb2187301287ac122 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b05 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.272457:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.272459:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845021.272464:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51e00/0xb2187301287ac122 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b05 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.272474:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51e00, rc 0) 00000100:00100000:0.0:1788845021.272498:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cefb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641427712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4010us (4510us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.272505:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 705 00000100:00100000:2.0:1788845021.276517:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.276578:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.276663:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641428096 00000100:00100000:3.0:1788845021.276675:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641428096 00000100:00100000:3.0:1788845021.276735:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 447 00000100:00100000:3.0:1788845021.276745:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641428096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.276765:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.276782:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac129 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b1a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.276840:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660192640:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.276952:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.277492:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641427968 00000100:00100000:1.0:1788845021.277497:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641427968 00000100:00100000:1.0:1788845021.277536:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 706 00000100:00100000:1.0:1788845021.277543:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641427968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.277556:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.277581:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac130 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.277635:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.277666:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660192768:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.277717:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.281419:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02484e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660192640:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845021.281458:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac129 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b1a expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.281481:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac129 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b1a expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.281511:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:3.0:1788845021.281565:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641428096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4822us (4998us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.281583:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 447 00000100:00100000:1.0:1788845021.281748:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660192768:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845021.281766:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1015808 00010000:00010000:1.0:1788845021.281777:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac130 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b13 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.281790:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.281793:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.281797:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac130 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b13 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.281807:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a400, rc 0) 00000100:00100000:1.0:1788845021.281826:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641427968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4284us (5322us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.281832:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 706 00000100:00100000:2.0:1788845021.287135:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.287217:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641428224 00000100:00100000:3.0:1788845021.287227:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641428224 00002000:00100000:3.0:1788845021.287285:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360e9500 x1875739641428224/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.287297:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360e9500 with x1875739641428224 00000100:00100000:3.0:1788845021.287333:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 404 00000100:00100000:3.0:1788845021.287343:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641428224:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845021.300450:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641428224:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 13099us (13315us total) trans 0 rc 65536/65536 00000100:00100000:0.0:1788845021.300466:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 404 00002000:00100000:0.0:1788845021.300479:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360e9500 x1875739641428224/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845021.300489:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360e9500 with x1875739641428224 00000100:00100000:3.0:1788845021.305555:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.305792:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641428352 00000100:00100000:1.0:1788845021.305802:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641428352 00002000:00100000:1.0:1788845021.305853:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02484e00 x1875739641428352/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.305863:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02484e00 with x1875739641428352 00000100:00100000:1.0:1788845021.305978:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 405 00000100:00100000:1.0:1788845021.306001:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02484e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641428352:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.311073:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02484e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641428352:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 5063us (5794us total) trans 0 rc 36864/36864 00000100:00100000:3.0:1788845021.311087:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 405 00002000:00100000:3.0:1788845021.311100:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02484e00 x1875739641428352/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.311110:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02484e00 with x1875739641428352 00000100:00100000:1.0:1788845021.314487:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.314731:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641428480 00000100:00100000:2.0:1788845021.314740:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641428480 00002000:00100000:2.0:1788845021.314788:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741880 x1875739641428480/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.314798:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741880 with x1875739641428480 00000100:00100000:2.0:1788845021.314909:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 406 00000100:00100000:2.0:1788845021.314992:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641428480:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845021.318504:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1015808 -> 1015808 00000004:00010000:3.0:1788845021.318513:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.318516:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845019 00000004:00010000:3.0:1788845021.318518:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.318521:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 134 -> 134 00000100:00100000:3.0:1788845021.319911:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641428480:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4980us (5424us total) trans 4294967591 rc 0/0 00000100:00100000:3.0:1788845021.319936:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 406 00002000:00100000:3.0:1788845021.319950:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741880 x1875739641428480/t4294967591(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/448 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.319960:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741880 with x1875739641428480 00000100:00100000:1.0:1788845021.323032:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.323130:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641428608 00000100:00100000:3.0:1788845021.323140:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641428608 00000100:00100000:3.0:1788845021.323235:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 707 00000100:00100000:3.0:1788845021.323248:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641428608:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845021.357549:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967591 is committed 00000100:00100000:3.0:1788845021.357866:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641428608:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 34616us (34838us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.357879:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 707 00000100:00100000:0.0:1788845021.369816:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.369873:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.370290:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641428736 00000100:00100000:2.0:1788845021.370302:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641428736 00000100:00100000:2.0:1788845021.370359:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 708 00000100:00100000:2.0:1788845021.370385:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641428736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.370409:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.370430:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac137 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b21 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.370468:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.370511:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360eb480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660192896:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.370722:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.372337:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641428864 00000100:00100000:0.0:1788845021.372348:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641428864 00000100:00100000:0.0:1788845021.372410:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 448 00000100:00100000:0.0:1788845021.372419:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641428864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.372435:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.372452:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000063efd0d5/0xb2187301287ac13e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b28 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.372524:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660193024:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.372607:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000004:00010000:0.0:1788845021.375364:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845019 -> 1788845021 00000004:00010000:0.0:1788845021.375373:0:5839:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 134 -> 262 00000100:00100000:0.0:1788845021.375396:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360eb480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660192896:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845021.375422:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1015808 00010000:00010000:0.0:1788845021.375440:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac137 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b21 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.375459:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.375462:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845021.375467:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac137 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b21 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.375480:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:0.0:1788845021.375537:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641428736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5154us (5737us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.375546:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 708 00000100:00100000:1.0:1788845021.378596:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660193024:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845021.378642:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000063efd0d5/0xb2187301287ac13e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b28 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.378667:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000063efd0d5/0xb2187301287ac13e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b28 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.378680:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51e00, rc 0) 00000100:00100000:1.0:1788845021.378740:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641428864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6321us (8865us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.378748:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 448 00000100:00100000:2.0:1788845021.381932:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.382066:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641428992 00000100:00100000:0.0:1788845021.382075:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641428992 00000100:00100000:0.0:1788845021.382136:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 709 00000100:00100000:0.0:1788845021.382147:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641428992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.382169:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.382190:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51800/0xb2187301287ac145 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b2f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.382246:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.382290:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660193152:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.382373:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.384318:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.384786:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641429120 00000100:00100000:0.0:1788845021.384795:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641429120 00000100:00100000:0.0:1788845021.384843:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 449 00000100:00100000:0.0:1788845021.384852:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641429120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.384868:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.384905:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e5105eb1/0xb2187301287ac14c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b36 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.385093:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a007eb5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660193280:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.385155:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.387678:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660193152:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845021.387709:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1015808 00010000:00010000:0.0:1788845021.387729:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51800/0xb2187301287ac145 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b2f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.387750:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.387753:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845021.387758:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51800/0xb2187301287ac145 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b2f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.387795:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51800, rc 0) 00000100:00100000:0.0:1788845021.387855:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641428992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5709us (5974us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.387864:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 709 00000100:00100000:0.0:1788845021.391436:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a007eb5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660193280:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845021.391478:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e5105eb1/0xb2187301287ac14c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b36 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.391505:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e5105eb1/0xb2187301287ac14c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b36 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.391518:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a50a00, rc 0) 00000100:00100000:0.0:1788845021.391592:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641429120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6739us (7281us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.391600:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 449 00000100:00100000:2.0:1788845021.395214:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.395373:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641429248 00000100:00100000:3.0:1788845021.395384:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641429248 00000100:00100000:3.0:1788845021.395443:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 450 00000100:00100000:3.0:1788845021.395455:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641429248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.395477:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.395497:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b3d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.395531:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.395540:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 10) 00010000:00010000:3.0:1788845021.395546:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 10) 00010000:00010000:3.0:1788845021.395672:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1114112->1142783] (req 1114112->1142783) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.395717:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.395738:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 10 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.395754:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660193408:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.395835:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.399190:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.399309:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660193408:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845021.399345:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1114112->1142783] (req 1114112->1142783) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.399368:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:3.0:1788845021.399426:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641429248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3973us (4221us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.399434:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 450 00000100:00100000:0.0:1788845021.401240:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641429376 00000100:00100000:0.0:1788845021.401384:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641429376 00010000:00010000:2.0:1788845021.401525:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 11 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.401675:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 256 00000100:00100000:2.0:1788845021.401686:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ea300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641429376:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845021.401703:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845021.401727:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 11 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.401745:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 11 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.401792:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000070017058/0xb2187301287ac0e3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7adb expref: 11 pid: 6684 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.401810:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 10) 00010000:00010000:2.0:1788845021.401814:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 10) 00010000:00010000:2.0:1788845021.401822:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.401832:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.401871:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.401914:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ead80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660193536:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845021.402000:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.404963:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ead80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660193536:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845021.404992:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845021.405052:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ea300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641429376:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3359us (5863us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.405065:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 256 00000100:00100000:0.0:1788845021.415522:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.415792:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641429504 00000100:00100000:3.0:1788845021.415818:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641429504 00002000:00100000:3.0:1788845021.415872:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1114112->1118207) req@ffff94a007eb5c00 x1875739641429504/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.415911:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb5c00 with x1875739641429504 ext(1114112->1118207) 00002000:00010000:3.0:1788845021.415918:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb5c00 00000100:00100000:3.0:1788845021.415960:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 256 00000100:00100000:3.0:1788845021.415970:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641429504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.416225:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641429504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (713us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845021.416233:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 256 00002000:00100000:3.0:1788845021.416241:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1114112->1118207) req@ffff94a007eb5c00 x1875739641429504/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.416250:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb5c00 with x1875739641429504 ext(1114112->1118207) 00002000:00010000:3.0:1788845021.416253:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb5c00 00000100:00100000:2.0:1788845021.420142:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.420249:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641429632 00000100:00100000:1.0:1788845021.420260:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641429632 00002000:00100000:1.0:1788845021.420311:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1138688->1142783) req@ffff94a0360e8700 x1875739641429632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.420323:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360e8700 with x1875739641429632 ext(1138688->1142783) 00002000:00010000:1.0:1788845021.420329:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360e8700 00000100:00100000:1.0:1788845021.420364:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 257 00000100:00100000:1.0:1788845021.420375:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641429632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.420634:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641429632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 260us (502us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845021.420656:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 257 00002000:00100000:1.0:1788845021.420663:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1138688->1142783) req@ffff94a0360e8700 x1875739641429632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.420672:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360e8700 with x1875739641429632 ext(1138688->1142783) 00002000:00010000:1.0:1788845021.420676:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360e8700 00000100:00100000:2.0:1788845021.424291:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.424401:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641429760 00000100:00100000:1.0:1788845021.424411:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641429760 00000100:00100000:1.0:1788845021.424467:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 710 00000100:00100000:1.0:1788845021.424476:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641429760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.424497:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.424519:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac15a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b44 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.424553:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.424701:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660193664:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.424782:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.426988:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660193664:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845021.427016:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1015808 00010000:00010000:1.0:1788845021.427036:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac15a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b44 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.427066:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.427070:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.427090:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac15a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b44 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.427106:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b600, rc 0) 00000100:00100000:1.0:1788845021.427167:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641429760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2690us (2884us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.427175:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 710 00000100:00100000:2.0:1788845021.430496:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.430745:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641429888 00000100:00100000:3.0:1788845021.430756:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641429888 00000100:00100000:3.0:1788845021.430814:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 711 00000100:00100000:3.0:1788845021.430825:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641429888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.430845:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.430867:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac161 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b4b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.431234:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.431291:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ead80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660193792:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.431383:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.434320:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ead80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660193792:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845021.434351:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1015808 00010000:00010000:0.0:1788845021.434371:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac161 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b4b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.434393:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.434410:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845021.434416:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac161 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b4b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.434430:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:0.0:1788845021.434491:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641429888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3667us (4005us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.434500:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 711 00000100:00100000:0.0:1788845021.437237:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.437328:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641430016 00000100:00100000:3.0:1788845021.437338:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641430016 00000100:00100000:3.0:1788845021.437399:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 451 00000100:00100000:3.0:1788845021.437409:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641430016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.437443:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.437460:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b52 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.437493:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.437501:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 10) 00010000:00010000:3.0:1788845021.437505:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 10) 00010000:00010000:3.0:1788845021.437511:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1331200->1392639] (req 1331200->1392639) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 11 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.437542:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.437560:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 7 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.437692:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660193920:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.437777:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.441287:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.441396:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660193920:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845021.441429:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1331200->1392639] (req 1331200->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 11 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.441452:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:3.0:1788845021.441529:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:14891:x1875739641430016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4121us (4303us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.441538:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 451 00000100:00100000:1.0:1788845021.441746:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641430144 00000100:00100000:1.0:1788845021.441754:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641430144 00002000:00100000:1.0:1788845021.441796:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1114112->1142783) req@ffff94a0360eb480 x1875739641430144/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.441807:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360eb480 with x1875739641430144 ext(1114112->1142783) 00010000:00010000:1.0:1788845021.441814:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 8 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.441834:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 8 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845021.441842:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a0360eb480 00000100:00100000:1.0:1788845021.441876:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 9 00000100:00100000:1.0:1788845021.441990:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641430144:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845021.447140:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641430144:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5245us (5852us total) trans 4294967381 rc 0/0 00000100:00100000:3.0:1788845021.447156:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 9 00002000:00100000:3.0:1788845021.447168:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1114112->1142783) req@ffff94a0360eb480 x1875739641430144/t4294967381(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/448 e 0 to 0 dl 1788845032 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.447180:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360eb480 with x1875739641430144 ext(1114112->1142783) 00010000:00010000:3.0:1788845021.447191:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.447215:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845021.447223:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a0360eb480 00000100:00100000:2.0:1788845021.450322:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.450775:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641430272 00000100:00100000:3.0:1788845021.450785:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641430272 00010000:00010000:3.0:1788845021.450827:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.450873:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 257 00000100:00100000:3.0:1788845021.451096:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008916850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641430272:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845021.451196:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845021.451223:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.451243:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.451327:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac153 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1114112->1142783) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b3d expref: 10 pid: 6682 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.451346:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 10) 00010000:00010000:2.0:1788845021.451350:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 10) 00010000:00010000:2.0:1788845021.451359:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.451369:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.451411:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.451426:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ead80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660194048:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845021.451548:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.458083:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.458425:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ead80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660194048:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845021.458466:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845021.458526:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008916850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641430272:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7644us (8274us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.458538:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 257 00000100:00100000:3.0:1788845021.458843:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641430400 00000100:00100000:3.0:1788845021.458851:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641430400 00000100:00100000:3.0:1788845021.458948:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 712 00000100:00100000:3.0:1788845021.458965:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641430400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.458986:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.459006:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b59 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.459035:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.459054:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.459087:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.459105:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 14 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.459118:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660194176:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.459168:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.463528:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660194176:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845021.463567:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.463591:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.463594:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.463600:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.463626:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:3.0:1788845021.463691:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641430400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4727us (5618us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.463700:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 712 00000100:00100000:1.0:1788845021.464080:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.465343:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641430528 00000100:00100000:2.0:1788845021.465356:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641430528 00010000:00010000:2.0:1788845021.465397:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 15 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.465438:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 258 00000100:00100000:2.0:1788845021.465448:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641430528:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845021.465461:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845021.465487:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 15 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.465504:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 15 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.465526:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac106 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7af7 expref: 15 pid: 5840 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.465543:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.465550:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.465690:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.465720:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.465759:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.465772:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ea300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660194304:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845021.465942:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.471230:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.471293:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641430656 00000100:00100000:3.0:1788845021.471303:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641430656 00000100:00100000:3.0:1788845021.471362:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 713 00000100:00100000:3.0:1788845021.471374:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034a0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641430656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.471449:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.471471:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ac176 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b60 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.471512:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.471552:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660194432:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.471642:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.473399:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ea300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660194304:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845021.473430:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845021.473544:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641430528:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8096us (9464us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845021.473587:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 258 00000004:00010000:3.0:1788845021.473876:0:11036:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 1015808 -> 1031310 00000100:00100000:3.0:1788845021.473945:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660194432:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845021.473961:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:3.0:1788845021.475169:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ac176 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b60 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.475203:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.475206:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.475227:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ac176 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b60 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.475245:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845021.475769:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034a0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641430656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4395us (4546us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.475779:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 713 00000100:00100000:2.0:1788845021.478859:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.479434:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641430784 00000100:00100000:1.0:1788845021.479445:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641430784 00000100:00100000:1.0:1788845021.479535:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 714 00000100:00100000:1.0:1788845021.479546:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641430784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.479702:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.479729:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac17d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b67 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.479769:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.479812:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660194560:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.480052:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.482756:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660194560:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845021.482787:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:1.0:1788845021.482806:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac17d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b67 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.482829:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.482832:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.482837:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac17d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b67 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.482871:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a400, rc 0) 00000100:00100000:1.0:1788845021.483142:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641430784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3596us (4291us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.483151:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 714 00000100:00100000:0.0:1788845021.487073:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.487195:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641430912 00000100:00100000:3.0:1788845021.487205:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641430912 00000100:00100000:2.0:1788845021.487430:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 407 00000100:00100000:2.0:1788845021.487441:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641430912:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845021.487750:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031310 -> 1015808 00000004:00010000:2.0:1788845021.487754:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845021.487757:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845021.487759:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845021.487761:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 262 -> 262 00000100:00100000:2.0:1788845021.487830:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641430912:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 391us (766us total) trans 4294967592 rc 0/0 00000100:00100000:2.0:1788845021.487837:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 407 00000100:00100000:3.0:1788845021.491104:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.491503:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641431040 00000100:00100000:0.0:1788845021.491513:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641431040 00000100:00100000:0.0:1788845021.491706:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 452 00000100:00100000:0.0:1788845021.491716:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641431040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.491735:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.491751:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac184 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b6e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.491808:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660194688:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.491926:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00002000:00010000:0.0:1788845021.494209:0:6685:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1253382 -> 1389356 00000100:00100000:0.0:1788845021.494241:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660194688:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845021.494284:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac184 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b6e expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.494326:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac184 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b6e expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.494342:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51c00, rc 0) 00000100:00100000:0.0:1788845021.494442:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641431040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2726us (3344us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.494450:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 452 00000100:00100000:2.0:1788845021.498370:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.498465:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641431168 00000100:00100000:0.0:1788845021.498476:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641431168 00000100:00100000:0.0:1788845021.498583:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 453 00000100:00100000:0.0:1788845021.498594:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641431168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.498613:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.498630:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac18b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b75 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.498691:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660194816:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.498774:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.502018:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660194816:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845021.502095:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac18b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b75 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.502122:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac18b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b75 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.502157:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a50600, rc 0) 00000100:00100000:0.0:1788845021.502270:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641431168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3676us (3907us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.502279:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 453 00000100:00100000:1.0:1788845021.506383:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.506479:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641431296 00000100:00100000:3.0:1788845021.506497:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641431296 00000100:00100000:3.0:1788845021.506722:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 454 00000100:00100000:3.0:1788845021.506734:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641431296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.506753:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.506772:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac192 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b7c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.506844:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660194944:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.507749:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.511082:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660194944:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845021.511119:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac192 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b7c expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.511145:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac192 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b7c expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.511158:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:3.0:1788845021.511264:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641431296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4531us (4889us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.511286:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 454 00000100:00100000:2.0:1788845021.514380:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.514464:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641431424 00000100:00100000:3.0:1788845021.514475:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641431424 00000100:00100000:3.0:1788845021.514591:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:3.0:1788845021.514602:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641431424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.514621:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.514641:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b83 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.514674:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.514684:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 10) 00010000:00010000:3.0:1788845021.514688:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:3.0:1788845021.514694:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1155072->1159167] (req 1155072->1159167) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.514728:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.514747:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 10 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.514760:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660195072:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.514836:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.518217:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660195072:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845021.518268:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1155072->1159167] (req 1155072->1159167) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.518293:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:3.0:1788845021.518369:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641431424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3767us (3996us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.518378:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:2.0:1788845021.520463:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.520554:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641431552 00000100:00100000:3.0:1788845021.520677:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641431552 00002000:00100000:3.0:1788845021.520740:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1331200->1389355) req@ffff949f01c49880 x1875739641431552/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.520751:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c49880 with x1875739641431552 ext(1331200->1389355) 00010000:00010000:3.0:1788845021.520761:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 11 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.520784:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 11 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845021.520794:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f01c49880 00000100:00100000:3.0:1788845021.520955:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 10 00000100:00100000:3.0:1788845021.520965:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641431552:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845021.525445:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641431552:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4470us (4981us total) trans 4294967382 rc 0/0 00000100:00100000:3.0:1788845021.525460:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 10 00002000:00100000:3.0:1788845021.525486:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1331200->1389355) req@ffff949f01c49880 x1875739641431552/t4294967382(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/448 e 0 to 0 dl 1788845032 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.525497:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c49880 with x1875739641431552 ext(1331200->1389355) 00010000:00010000:3.0:1788845021.525508:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 13 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.525529:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 13 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845021.525537:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f01c49880 00000100:00100000:2.0:1788845021.529816:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.529918:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641431680 00000100:00100000:0.0:1788845021.529935:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641431680 00010000:00010000:0.0:1788845021.529974:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 13 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.530011:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 259 00000100:00100000:0.0:1788845021.530020:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641431680:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845021.530034:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845021.530064:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 13 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.530083:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 13 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.530107:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ac168 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1331200->1392639) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b52 expref: 13 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.530124:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:0.0:1788845021.530128:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:0.0:1788845021.530137:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.530160:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.530196:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.530210:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660195200:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845021.530295:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.535455:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.535514:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.535605:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641431936 00000100:00100000:3.0:1788845021.535615:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641431936 00002000:00100000:3.0:1788845021.535665:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f024d4380 x1875739641431936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.535676:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f024d4380 with x1875739641431936 00000100:00100000:3.0:1788845021.535758:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 408 00000100:00100000:3.0:1788845021.535768:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641431936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.536076:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641431936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 307us (629us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.536084:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 408 00002000:00100000:3.0:1788845021.536092:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f024d4380 x1875739641431936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.536099:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f024d4380 with x1875739641431936 00000100:00100000:3.0:1788845021.536125:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641431808 00000100:00100000:3.0:1788845021.536129:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641431808 00002000:00100000:3.0:1788845021.536159:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1392639) req@ffff949f024d5180 x1875739641431808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.536168:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d5180 with x1875739641431808 ext(1048576->1392639) 00002000:00010000:3.0:1788845021.536173:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d5180 00000100:00100000:3.0:1788845021.536191:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 258 00000100:00100000:3.0:1788845021.536198:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f024d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641431808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845021.537621:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660195200:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845021.537648:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845021.537691:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641431680:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7671us (7884us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.537700:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 259 00000100:00100000:3.0:1788845021.543642:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f024d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641431808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 7433us (8117us total) trans 0 rc 344064/344064 00000100:00100000:3.0:1788845021.543656:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 258 00002000:00100000:3.0:1788845021.543668:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1392639) req@ffff949f024d5180 x1875739641431808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 344064/344064 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.543679:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f024d5180 with x1875739641431808 ext(1048576->1392639) 00002000:00010000:3.0:1788845021.543686:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f024d5180 00000100:00100000:0.0:1788845021.548262:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.548445:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641432064 00000100:00100000:2.0:1788845021.548455:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641432064 00000100:00100000:2.0:1788845021.548518:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 715 00000100:00100000:2.0:1788845021.548529:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641432064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.548552:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.548585:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287ac1a0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b8a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.548625:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.548672:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02483100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660195328:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.548758:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.551426:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000004:00010000:3.0:1788845021.551593:0:5840:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 1015808 -> 1031310 00000100:00100000:3.0:1788845021.551621:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02483100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660195328:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845021.551645:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:3.0:1788845021.551664:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287ac1a0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b8a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.551685:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.551689:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.551694:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287ac1a0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b8a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.551709:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865e00, rc 0) 00000100:00100000:3.0:1788845021.551780:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641432064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3253us (3528us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.551789:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 715 00000100:00100000:0.0:1788845021.552056:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641432192 00000100:00100000:0.0:1788845021.552064:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641432192 00000100:00100000:0.0:1788845021.552114:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 456 00000100:00100000:0.0:1788845021.552124:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2369:x1875739641432192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.552139:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.552155:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b91 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.552181:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:0.0:1788845021.552186:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:0.0:1788845021.552203:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.552220:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.552232:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51c00, rc 0) 00000100:00100000:0.0:1788845021.552256:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641432192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 134us (839us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.552262:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 456 00000100:00100000:3.0:1788845021.555534:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.555744:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641432320 00000100:00100000:2.0:1788845021.555755:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641432320 00000100:00100000:2.0:1788845021.555813:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 716 00000100:00100000:2.0:1788845021.555824:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641432320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.555845:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.555866:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287ac1ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b98 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.556111:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.556155:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660195456:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.556253:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.558941:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660195456:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845021.558969:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:3.0:1788845021.558987:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287ac1ae lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b98 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.559010:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.559013:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.559019:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287ac1ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b98 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.559034:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:3.0:1788845021.559103:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641432320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3279us (3578us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.559111:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 716 00000100:00100000:3.0:1788845021.563002:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.563147:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641432448 00000100:00100000:0.0:1788845021.563157:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641432448 00000100:00100000:0.0:1788845021.563217:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 457 00000100:00100000:0.0:1788845021.563227:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641432448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.563246:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.563265:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7b9f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.563296:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.563305:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.563311:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:0.0:1788845021.563315:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:0.0:1788845021.563336:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1568768->1626111] (req 1568768->1626111) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.563370:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.563389:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 14 pid: 6685 timeout: 510 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.563402:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660195584:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845021.563488:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.563498:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 9 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.563507:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660195712:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845021.563530:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.566467:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660195584:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845021.566485:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660195712:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845021.566523:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1568768->1626111] (req 1568768->1626111) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.566548:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a50600, rc 0) 00000100:00100000:0.0:1788845021.566637:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641432448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3410us (3644us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.566646:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 457 00000100:00100000:2.0:1788845021.570329:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.570365:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.570457:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641432576 00000100:00100000:1.0:1788845021.570467:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641432576 00010000:00010000:1.0:1788845021.570508:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 14 pid: 6685 timeout: 510 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.570549:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641432704 00000100:00100000:1.0:1788845021.570553:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641432704 00010000:00010000:1.0:1788845021.570578:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 10 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.570609:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 260 00000100:00100000:1.0:1788845021.570619:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641432576:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845021.570635:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845021.570657:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 14 pid: 6685 timeout: 510 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.570676:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 14 pid: 6685 timeout: 510 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.570700:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085d7e3fa/0xb2187301287ac1a7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b91 expref: 14 pid: 6685 timeout: 510 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.570734:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845021.570781:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641432576:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 163us (462us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845021.570789:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 260 00000100:00100000:1.0:1788845021.570808:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 261 00000100:00100000:1.0:1788845021.570815:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641432704:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845021.570824:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845021.570833:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 10 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.570845:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 10 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.570859:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac199 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1159167) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b83 expref: 10 pid: 7983 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.570869:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:1.0:1788845021.570873:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 10) 00010000:00010000:1.0:1788845021.570931:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.570944:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.570992:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.571006:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660195840:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845021.571067:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.576145:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.576267:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660195840:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845021.576293:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845021.576354:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641432704:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5538us (5987us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845021.576366:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 261 00000100:00100000:0.0:1788845021.576817:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641432832 00000100:00100000:0.0:1788845021.576825:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641432832 00000100:00100000:0.0:1788845021.576871:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 458 00000100:00100000:0.0:1788845021.576922:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641432832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.576941:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.576961:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a464dfc0/0xb2187301287ac1bc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ba6 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.577016:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660195968:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.577094:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00002000:00010000:0.0:1788845021.582112:0:6685:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1389356 -> 1624774 00000100:00100000:0.0:1788845021.582149:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660195968:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845021.582191:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a464dfc0/0xb2187301287ac1bc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ba6 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.582214:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a464dfc0/0xb2187301287ac1bc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ba6 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.582227:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51600, rc 0) 00000100:00100000:0.0:1788845021.582305:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641432832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5425us (6168us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.582314:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 458 00000100:00100000:2.0:1788845021.586736:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.586863:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641432960 00000100:00100000:3.0:1788845021.586873:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641432960 00000100:00100000:3.0:1788845021.586968:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 459 00000100:00100000:3.0:1788845021.586979:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641432960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.586999:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.587019:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac1c3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bad expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.587117:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660196096:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.587204:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.590653:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660196096:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845021.590697:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac1c3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bad expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.590723:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac1c3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bad expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.590737:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:3.0:1788845021.590794:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641432960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3816us (4066us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.590804:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 459 00000100:00100000:2.0:1788845021.593944:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.594120:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641433088 00000100:00100000:3.0:1788845021.594130:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641433088 00000100:00100000:3.0:1788845021.594185:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 460 00000100:00100000:3.0:1788845021.594195:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641433088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.594212:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.594229:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac1ca lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bb4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.594384:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660196224:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.594502:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.597209:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660196224:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845021.597250:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac1ca lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bb4 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.597423:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac1ca lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bb4 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.597436:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:3.0:1788845021.597526:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641433088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3332us (3590us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.597536:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 460 00000100:00100000:2.0:1788845021.600351:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.600437:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641433216 00000100:00100000:0.0:1788845021.600446:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641433216 00000100:00100000:0.0:1788845021.600531:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 461 00000100:00100000:0.0:1788845021.600542:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641433216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.600685:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.600706:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a464dfc0/0xb2187301287ac1d1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bbb expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.600770:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660196352:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.600914:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.603836:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660196352:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845021.603879:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a464dfc0/0xb2187301287ac1d1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bbb expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.604033:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a464dfc0/0xb2187301287ac1d1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bbb expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.604057:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51600, rc 0) 00000100:00100000:0.0:1788845021.604153:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641433216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3612us (3810us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.604162:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 461 00000100:00100000:2.0:1788845021.608190:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.608463:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641433344 00000100:00100000:0.0:1788845021.608473:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641433344 00000100:00100000:0.0:1788845021.608584:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 462 00000100:00100000:0.0:1788845021.608595:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641433344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.608630:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.608648:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa80d4a9/0xb2187301287ac1d8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bc2 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.608711:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660196480:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.608796:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.612192:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660196480:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845021.612240:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fa80d4a9/0xb2187301287ac1d8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bc2 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.612265:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa80d4a9/0xb2187301287ac1d8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bc2 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.612278:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51400, rc 0) 00000100:00100000:0.0:1788845021.612374:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641433344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3779us (4191us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845021.612383:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 462 00000100:00100000:2.0:1788845021.616098:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.616191:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641433472 00000100:00100000:3.0:1788845021.616201:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641433472 00002000:00100000:3.0:1788845021.616253:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482a00 x1875739641433472/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.616331:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482a00 with x1875739641433472 00000100:00100000:3.0:1788845021.616399:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 409 00000100:00100000:3.0:1788845021.616410:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641433472:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.616808:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641433472:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 398us (719us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845021.616816:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 409 00002000:00100000:3.0:1788845021.616824:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482a00 x1875739641433472/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.616832:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482a00 with x1875739641433472 00000100:00100000:2.0:1788845021.620834:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.621078:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641433600 00000100:00100000:3.0:1788845021.621089:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641433600 00002000:00100000:2.0:1788845021.621211:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481180 x1875739641433600/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.621223:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481180 with x1875739641433600 00000100:00100000:2.0:1788845021.621289:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 410 00000100:00100000:2.0:1788845021.621299:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641433600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845021.621522:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641433600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 223us (698us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845021.621530:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 410 00002000:00100000:2.0:1788845021.621538:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481180 x1875739641433600/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.621546:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481180 with x1875739641433600 00000100:00100000:0.0:1788845021.624711:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.624815:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641433728 00000100:00100000:1.0:1788845021.624826:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641433728 00002000:00100000:1.0:1788845021.624875:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d180 x1875739641433728/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.624984:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d180 with x1875739641433728 00000100:00100000:1.0:1788845021.625022:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 411 00000100:00100000:1.0:1788845021.625032:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641433728:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.625247:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641433728:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 216us (545us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845021.625256:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 411 00002000:00100000:1.0:1788845021.625263:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d180 x1875739641433728/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.625271:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d180 with x1875739641433728 00000100:00100000:0.0:1788845021.628486:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.628850:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641433856 00000100:00100000:1.0:1788845021.628860:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641433856 00002000:00100000:1.0:1788845021.628951:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358e300 x1875739641433856/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.628962:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358e300 with x1875739641433856 00000100:00100000:1.0:1788845021.628999:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 412 00000100:00100000:1.0:1788845021.629009:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641433856:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.629240:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641433856:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (763us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845021.629248:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 412 00002000:00100000:1.0:1788845021.629256:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358e300 x1875739641433856/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.629264:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358e300 with x1875739641433856 00000100:00100000:0.0:1788845021.633142:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.633236:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641433984 00000100:00100000:3.0:1788845021.633247:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641433984 00002000:00100000:3.0:1788845021.633295:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ed80 x1875739641433984/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.633306:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ed80 with x1875739641433984 00000100:00100000:3.0:1788845021.633343:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 413 00000100:00100000:3.0:1788845021.633353:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641433984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.633732:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641433984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 380us (599us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845021.633741:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 413 00002000:00100000:3.0:1788845021.633748:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ed80 x1875739641433984/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.633757:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ed80 with x1875739641433984 00000100:00100000:0.0:1788845021.637748:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.637849:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641434112 00000100:00100000:1.0:1788845021.637859:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641434112 00002000:00100000:1.0:1788845021.638026:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ea00 x1875739641434112/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.638037:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ea00 with x1875739641434112 00000100:00100000:1.0:1788845021.638138:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 414 00000100:00100000:1.0:1788845021.638149:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641434112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.638443:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641434112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 296us (707us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845021.638452:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 414 00002000:00100000:1.0:1788845021.638459:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ea00 x1875739641434112/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.638487:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ea00 with x1875739641434112 00000100:00100000:1.0:1788845021.641867:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.642113:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641434240 00000100:00100000:2.0:1788845021.642124:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641434240 00002000:00100000:2.0:1788845021.642175:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035742a00 x1875739641434240/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.642186:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742a00 with x1875739641434240 00000100:00100000:2.0:1788845021.642276:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 415 00000100:00100000:2.0:1788845021.642286:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641434240:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845021.642555:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641434240:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 270us (700us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845021.642575:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 415 00002000:00100000:2.0:1788845021.642582:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035742a00 x1875739641434240/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.642590:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742a00 with x1875739641434240 00000100:00100000:0.0:1788845021.645835:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.646455:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641434368 00000100:00100000:2.0:1788845021.646468:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641434368 00002000:00100000:2.0:1788845021.646515:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358f800 x1875739641434368/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.646525:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358f800 with x1875739641434368 00000100:00100000:2.0:1788845021.646635:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 416 00000100:00100000:2.0:1788845021.646647:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641434368:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845021.651120:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031310 -> 1031310 00000004:00010000:1.0:1788845021.651130:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:1.0:1788845021.651132:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:1.0:1788845021.651135:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:1.0:1788845021.651138:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 262 -> 262 00000100:00100000:1.0:1788845021.651223:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641434368:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4573us (5392us total) trans 4294967593 rc 0/0 00000100:00100000:1.0:1788845021.651235:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 416 00002000:00100000:1.0:1788845021.651249:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358f800 x1875739641434368/t4294967593(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/448 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.651360:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358f800 with x1875739641434368 00000100:00100000:2.0:1788845021.657122:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.657209:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641434496 00000100:00100000:3.0:1788845021.657219:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641434496 00000100:00100000:3.0:1788845021.657278:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 717 00000100:00100000:3.0:1788845021.657289:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641434496:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:3.0:1788845021.692009:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641434496:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 34709us (34883us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.692025:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 717 00000001:00080000:1.0:1788845021.692113:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967593 is committed 00000100:00100000:1.0:1788845021.695002:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.695099:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641434624 00000100:00100000:2.0:1788845021.695110:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641434624 00000100:00100000:2.0:1788845021.695174:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 718 00000100:00100000:2.0:1788845021.695184:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641434624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.695206:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.695226:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287ac1df lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bc9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.695268:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.695328:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660196608:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.695412:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b000 going to sleep for 11 seconds 00000004:00010000:1.0:1788845021.697967:0:5841:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 262 -> 326 00000100:00100000:1.0:1788845021.698001:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660196608:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845021.698027:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:1.0:1788845021.698056:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287ac1df lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bc9 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.698078:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.698081:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.698086:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287ac1df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bc9 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.698120:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865e00, rc 0) 00000100:00100000:1.0:1788845021.698171:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641434624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2989us (3177us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.698179:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 718 00000100:00100000:1.0:1788845021.701201:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.701300:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641434752 00000100:00100000:3.0:1788845021.701312:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641434752 00000100:00100000:3.0:1788845021.701376:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 719 00000100:00100000:3.0:1788845021.701386:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641434752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.701420:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.701442:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ac1e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bd0 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.701477:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.701517:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660196736:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.701611:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.703933:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660196736:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845021.703963:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:1.0:1788845021.703983:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ac1e6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd0 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.704005:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.704007:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.704012:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ac1e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd0 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.704027:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845021.704094:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641434752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2709us (2902us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.704103:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 719 00000100:00100000:3.0:1788845021.708328:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.708411:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641435008 00000100:00100000:2.0:1788845021.708421:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641435008 00000100:00100000:2.0:1788845021.708480:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 720 00000100:00100000:2.0:1788845021.708490:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641435008:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845021.708544:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845021.708584:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac0f1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ae2 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.708602:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.708607:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845021.708869:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.709171:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.709181:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.709215:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.709234:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 10 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.709270:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660196864:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845021.709362:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.712769:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.712951:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641435136 00000100:00100000:0.0:1788845021.712962:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641435136 00010000:00010000:0.0:1788845021.713003:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 11 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.713067:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 262 00000100:00100000:0.0:1788845021.713076:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641435136:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845021.713092:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845021.713099:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 11 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.713117:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 11 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.713139:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.713144:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.713153:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.713161:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.713179:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845021.713232:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641435136:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 157us (472us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.713239:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 262 00010000:00010000:2.0:1788845021.713905:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ac114 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7afe expref: 10 pid: 5841 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.713929:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660196864:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845021.713955:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.713964:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.713969:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845021.714098:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967594 00000004:00010000:2.0:1788845021.714150:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031310 -> 1031310 00000004:00010000:2.0:1788845021.714155:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845021.714158:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845021.714160:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845021.714162:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:2.0:1788845021.714170:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000125b3684 reply state 00000000e82c52cb transno 4294967595 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.714179:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845021.714283:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641435008:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5793us (5964us total) trans 4294967595 rc 0/0 00000100:00100000:2.0:1788845021.714303:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 720 00000020:00080000:2.0:1788845021.714308:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845021.721176:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.721194:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.721216:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.721223:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845021.721227:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.721235:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.721251:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.721255:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845021.721259:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.721265:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ac1ed lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.722821:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.723487:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641435264 00000100:00100000:3.0:1788845021.723498:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641435264 00000100:00100000:3.0:1788845021.723666:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 721 00000100:00100000:3.0:1788845021.723680:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641435264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.723705:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.723727:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bd7 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.723756:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.723764:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.723797:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.723831:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 9 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.723845:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cb4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660196992:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.724164:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.726267:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cb4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660196992:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845021.726300:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.726318:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845021.726321:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845021.726326:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.726336:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:0.0:1788845021.726388:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641435264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2710us (3574us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.726404:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 721 00000100:00100000:3.0:1788845021.728747:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.728928:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641435392 00000100:00100000:2.0:1788845021.728938:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641435392 00010000:00010000:2.0:1788845021.728978:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.729011:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 263 00000100:00100000:2.0:1788845021.729021:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cb4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641435392:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845021.729035:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845021.729073:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.729091:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.729112:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ac16f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b59 expref: 10 pid: 5839 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.729126:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.729131:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.729142:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.729151:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.729200:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.729214:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660197120:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845021.729294:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.732987:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.733172:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641435520 00000100:00100000:1.0:1788845021.733182:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641435520 00002000:00010000:1.0:1788845021.733223:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1061925->18446744073709551615) 00002000:00100000:1.0:1788845021.733228:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cb7800 with x1875739641435520 ext(1061925->18446744073709551615) 00002000:00010000:1.0:1788845021.733240:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cb7800. 00000100:00100000:1.0:1788845021.733290:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 259 00000100:00100000:1.0:1788845021.733300:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641435520:12345-192.168.201.29@tcp:10:fsx.0 00000100:00100000:2.0:1788845021.734293:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660197120:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845021.734315:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845021.734370:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cb4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641435392:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5351us (5630us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.734383:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 263 00002000:00010000:1.0:1788845021.735586:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1624774 -> 1061925 00002000:00010000:1.0:1788845021.735593:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00002000:00010000:1.0:1788845021.735596:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00002000:00010000:1.0:1788845021.735598:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00002000:00010000:1.0:1788845021.735600:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 341 -> 341 00000100:00100000:1.0:1788845021.735659:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641435520:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 2357us (2678us total) trans 4294967383 rc 0/0 00000100:00100000:1.0:1788845021.735670:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 259 00002000:00010000:1.0:1788845021.735675:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1061925->18446744073709551615) 00002000:00100000:1.0:1788845021.735679:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cb7800 with x1875739641435520 ext(1061925->18446744073709551615) 00002000:00010000:1.0:1788845021.735685:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cb7800. 00000100:00100000:3.0:1788845021.739608:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.739766:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641435648 00000100:00100000:1.0:1788845021.739777:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641435648 00000100:00100000:1.0:1788845021.739842:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 722 00000100:00100000:1.0:1788845021.739853:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cb5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641435648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.739877:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.739945:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac1fb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bde expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.740067:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.740085:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.740125:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845021.740160:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1061925 00000004:00200000:1.0:1788845021.740165:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845021.740190:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.740223:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac1fb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.740241:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.740250:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.740253:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845021.740256:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a400, rc 0) 00000100:00100000:1.0:1788845021.740324:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cb5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641435648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 473us (726us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845021.740332:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 722 00000100:00100000:3.0:1788845021.744550:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.744740:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641435776 00000100:00100000:1.0:1788845021.744750:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641435776 00000100:00100000:1.0:1788845021.744806:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 723 00000100:00100000:1.0:1788845021.744818:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641435776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.744838:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.744859:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ac209 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7be5 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.744950:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.744991:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660197248:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.745080:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.747803:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.748046:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660197248:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845021.748072:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:1.0:1788845021.748093:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ac209 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7be5 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.748114:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.748117:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.748123:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ac209 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7be5 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.748137:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b000, rc 0) 00000100:00100000:1.0:1788845021.748197:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641435776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3381us (3656us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.748205:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 723 00000100:00100000:2.0:1788845021.748411:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641435904 00000100:00100000:2.0:1788845021.748418:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641435904 00000100:00100000:2.0:1788845021.748468:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 463 00000100:00100000:2.0:1788845021.748477:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641435904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.748491:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.748505:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac210 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bec expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.748557:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660197376:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.748604:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.752268:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660197376:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845021.752309:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac210 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bec expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.752354:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac210 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bec expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.752368:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864000, rc 0) 00000100:00100000:2.0:1788845021.752428:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641435904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3952us (4632us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.752436:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 463 00000100:00100000:3.0:1788845021.756737:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.756781:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.756861:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641436160 00000100:00100000:2.0:1788845021.756871:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641436160 00000100:00100000:2.0:1788845021.756968:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 464 00000100:00100000:2.0:1788845021.756979:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641436160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.756998:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.757016:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287ac217 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bfa expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.757083:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660197504:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.757161:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.757448:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641436032 00000100:00100000:1.0:1788845021.757454:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641436032 00000100:00100000:1.0:1788845021.757495:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 724 00000100:00100000:1.0:1788845021.757503:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641436032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.757514:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.757527:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ac21e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7bf3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.757586:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.757612:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660197632:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.757641:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.761092:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660197632:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845021.761122:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:1.0:1788845021.761141:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ac21e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bf3 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.761163:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.761166:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.761171:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ac21e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bf3 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.761185:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b000, rc 0) 00000100:00100000:1.0:1788845021.761241:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641436032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3739us (4515us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.761249:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 724 00000100:00100000:2.0:1788845021.761423:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660197504:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845021.761443:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287ac217 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bfa expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.761458:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287ac217 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bfa expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.761485:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864400, rc 0) 00000100:00100000:2.0:1788845021.761505:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641436160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4529us (4724us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.761512:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 464 00000100:00100000:1.0:1788845021.766401:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.766474:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641436288 00000100:00100000:0.0:1788845021.766485:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641436288 00000100:00100000:0.0:1788845021.766542:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 725 00000100:00100000:0.0:1788845021.766551:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641436288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.766582:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.766602:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51800/0xb2187301287ac225 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c01 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.766638:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.766680:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660197760:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.766767:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fe40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.769808:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.770232:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660197760:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845021.770261:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:3.0:1788845021.770280:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51800/0xb2187301287ac225 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c01 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.770299:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.770302:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.770307:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02a51800/0xb2187301287ac225 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c01 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.770327:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a51800, rc 0) 00000100:00100000:3.0:1788845021.770383:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641436288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3833us (3992us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845021.770406:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 725 00000100:00100000:0.0:1788845021.771497:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641436416 00000100:00100000:0.0:1788845021.771509:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641436416 00000100:00100000:2.0:1788845021.771652:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 465 00000100:00100000:2.0:1788845021.771664:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641436416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.771681:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.771699:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287ac22c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c08 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.771757:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660197888:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.771834:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.774346:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660197888:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845021.774388:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287ac22c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c08 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.774412:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287ac22c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c08 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.774425:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864400, rc 0) 00000100:00100000:2.0:1788845021.774483:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641436416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2820us (4683us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.774492:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 465 00000100:00100000:3.0:1788845021.778318:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.778362:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.778482:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641436672 00000100:00100000:0.0:1788845021.778492:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641436672 00000100:00100000:0.0:1788845021.778555:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 466 00000100:00100000:0.0:1788845021.778577:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641436672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845021.778595:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845021.778614:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e5105eb1/0xb2187301287ac233 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c16 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845021.778671:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660198016:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845021.778760:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.779510:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641436544 00000100:00100000:2.0:1788845021.779517:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641436544 00000100:00100000:2.0:1788845021.779564:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 726 00000100:00100000:2.0:1788845021.779572:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641436544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.779586:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.779599:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287ac23a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c0f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.779629:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.779659:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660198144:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.779709:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00010000:00010000:1.0:1788845021.779764:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845021.781353:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660198016:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845021.781391:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e5105eb1/0xb2187301287ac233 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c16 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.781437:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e5105eb1/0xb2187301287ac233 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c16 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.781453:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02a50a00, rc 0) 00000100:00100000:1.0:1788845021.781507:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641436672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2933us (3144us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.781515:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 466 00000100:00100000:2.0:1788845021.785430:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660198144:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845021.785459:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031310 00010000:00010000:2.0:1788845021.785477:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287ac23a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c0f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.785499:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.785501:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845021.785506:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287ac23a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c0f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.785519:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864000, rc 0) 00000100:00100000:2.0:1788845021.785583:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641436544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6011us (7274us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.785591:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 726 00000100:00100000:2.0:1788845021.789225:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.789310:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641436800 00000100:00100000:1.0:1788845021.789321:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641436800 00000100:00100000:1.0:1788845021.789385:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 727 00000100:00100000:1.0:1788845021.789395:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641436800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.789419:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.789441:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c1d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.789470:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.789480:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c1d expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.789514:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.789531:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 15 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.789560:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660198272:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845021.789639:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.792445:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660198272:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845021.792483:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c1d expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.792501:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845021.792504:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845021.792509:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c1d expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.792519:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b400, rc 0) 00000100:00100000:1.0:1788845021.792674:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641436800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3279us (3458us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845021.792683:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 727 00000100:00100000:0.0:1788845021.795210:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.795489:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641436928 00000100:00100000:2.0:1788845021.795500:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641436928 00010000:00010000:2.0:1788845021.795537:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 16 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.795584:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 264 00000100:00100000:2.0:1788845021.795595:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008916050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641436928:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845021.795608:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845021.795634:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 16 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.795651:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 16 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.795672:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac1f4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bd7 expref: 16 pid: 11036 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.795687:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.795694:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c1d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.795705:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c1d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.795714:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c1d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.795767:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c1d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.795781:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660198400:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845021.795867:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.801447:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660198400:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845021.801474:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845021.801526:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008916050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641436928:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5933us (6325us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.801537:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 264 00000100:00100000:0.0:1788845021.803685:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.803801:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641437056 00000100:00100000:3.0:1788845021.803811:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641437056 00002000:00100000:3.0:1788845021.803856:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d500 x1875739641437056/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.803867:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d500 with x1875739641437056 00000100:00100000:3.0:1788845021.803940:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 417 00000100:00100000:3.0:1788845021.803949:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641437056:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.826984:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641437056:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 23022us (23294us total) trans 0 rc 1032192/1032192 00000100:00100000:1.0:1788845021.826999:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 417 00002000:00100000:1.0:1788845021.827013:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d500 x1875739641437056/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 1032192/1032192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.827024:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d500 with x1875739641437056 00000100:00100000:0.0:1788845021.831115:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.831664:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641437184 00000100:00100000:2.0:1788845021.831676:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641437184 00000100:00100000:2.0:1788845021.831789:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 467 00000100:00100000:2.0:1788845021.831802:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641437184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.831822:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.831838:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287ac248 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c24 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.831990:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660198528:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.832087:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.834413:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660198528:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845021.834454:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287ac248 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c24 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.834482:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287ac248 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c24 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.834496:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865400, rc 0) 00000100:00100000:2.0:1788845021.834667:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641437184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2866us (3560us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.834675:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 467 00000100:00100000:0.0:1788845021.837658:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.837757:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641437312 00000100:00100000:1.0:1788845021.837769:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641437312 00000100:00100000:1.0:1788845021.837858:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 468 00000100:00100000:1.0:1788845021.838004:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641437312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.838025:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.838054:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287ac24f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c2b expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.838113:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660198656:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845021.838190:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.840863:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660198656:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845021.841076:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287ac24f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c2b expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.841104:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e50c0968/0xb2187301287ac24f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c2b expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.841118:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b600, rc 0) 00000100:00100000:1.0:1788845021.841209:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641437312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3328us (3561us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845021.841217:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 468 00000100:00100000:1.0:1788845021.844386:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.844479:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641437440 00000100:00100000:2.0:1788845021.844490:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641437440 00000100:00100000:2.0:1788845021.844655:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 728 00000100:00100000:2.0:1788845021.844666:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641437440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.844686:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.844706:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c32 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.844736:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.844754:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.844767:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.844770:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845021.844775:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845021.844879:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641437440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 215us (502us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.844924:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 728 00000100:00100000:1.0:1788845021.848296:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.848388:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641437568 00000100:00100000:3.0:1788845021.848398:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641437568 00002000:00100000:3.0:1788845021.848452:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d3800 x1875739641437568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.848464:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d3800 with x1875739641437568 00000100:00100000:3.0:1788845021.848524:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 418 00000100:00100000:3.0:1788845021.848533:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641437568:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845021.853396:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641437568:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4852us (5098us total) trans 0 rc 65536/65536 00000100:00100000:2.0:1788845021.853410:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 418 00002000:00100000:2.0:1788845021.853422:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d3800 x1875739641437568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.853468:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d3800 with x1875739641437568 00000100:00100000:3.0:1788845021.856729:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.856824:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641437696 00000100:00100000:2.0:1788845021.856834:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641437696 00000100:00100000:2.0:1788845021.856924:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 469 00000100:00100000:2.0:1788845021.856934:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641437696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845021.856954:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845021.856974:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000382936ba/0xb2187301287ac25d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c39 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.857050:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660198784:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845021.857129:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.859337:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660198784:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845021.859375:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000382936ba/0xb2187301287ac25d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c39 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.859400:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000382936ba/0xb2187301287ac25d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c39 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.859414:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864e00, rc 0) 00000100:00100000:2.0:1788845021.859476:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641437696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2543us (2761us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.859485:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 469 00000100:00100000:1.0:1788845021.862747:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.862832:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641437824 00000100:00100000:3.0:1788845021.862857:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641437824 00000100:00100000:3.0:1788845021.862957:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 470 00000100:00100000:3.0:1788845021.862968:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641437824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.862985:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.863002:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac264 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c40 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.863070:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660198912:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845021.863148:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.865769:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660198912:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845021.865807:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac264 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c40 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.865847:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac264 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c40 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.865860:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:2.0:1788845021.866025:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641437824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3059us (3288us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845021.866034:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 470 00000100:00100000:2.0:1788845021.869837:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.869938:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641437952 00000100:00100000:3.0:1788845021.869948:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641437952 00000100:00100000:3.0:1788845021.870010:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 729 00000100:00100000:3.0:1788845021.870021:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641437952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.870052:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.870073:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac26b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c47 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.870161:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac272 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.870176:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac272 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.870186:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac272 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845021.870211:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1061925 00000004:00200000:3.0:1788845021.870214:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845021.870231:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac272 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.870287:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac26b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c47 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.870305:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac272 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c47 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.870314:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.870317:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845021.870320:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:3.0:1788845021.870379:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641437952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 358us (551us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.870385:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 729 00000100:00100000:1.0:1788845021.875419:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.875517:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641438080 00000100:00100000:2.0:1788845021.875527:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641438080 00002000:00100000:2.0:1788845021.875646:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dd500 x1875739641438080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.875659:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dd500 with x1875739641438080 00000100:00100000:2.0:1788845021.875732:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 419 00000100:00100000:2.0:1788845021.875743:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641438080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845021.885210:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641438080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 9459us (9790us total) trans 0 rc 65536/65536 00000100:00100000:0.0:1788845021.885224:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 419 00002000:00100000:0.0:1788845021.885238:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dd500 x1875739641438080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845021.885249:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dd500 with x1875739641438080 00000100:00100000:1.0:1788845021.888121:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.888198:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641438208 00000100:00100000:2.0:1788845021.888208:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641438208 00002000:00100000:2.0:1788845021.888256:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dea00 x1875739641438208/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845021.888267:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dea00 with x1875739641438208 00000100:00100000:2.0:1788845021.888336:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 420 00000100:00100000:2.0:1788845021.888346:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641438208:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.892108:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641438208:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 3752us (3987us total) trans 0 rc 36864/36864 00000100:00100000:3.0:1788845021.892121:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 420 00002000:00100000:3.0:1788845021.892134:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dea00 x1875739641438208/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.892145:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dea00 with x1875739641438208 00000100:00100000:1.0:1788845021.895459:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.895570:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641438336 00000100:00100000:3.0:1788845021.895579:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641438336 00000100:00100000:3.0:1788845021.895672:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:3.0:1788845021.895682:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641438336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.895702:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.895721:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c4e expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.895771:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 13 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.895779:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(10 10) 00010000:00010000:3.0:1788845021.895784:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 10) 00010000:00010000:3.0:1788845021.895790:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1544192->1581055] (req 1544192->1581055) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.895824:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 13 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.895842:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 13 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.895855:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660199040:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.895956:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.897815:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660199040:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845021.897851:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1544192->1581055] (req 1544192->1581055) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.897872:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845021.897957:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641438336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2277us (2506us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.897965:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:0.0:1788845021.899999:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.900076:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641438464 00000100:00100000:3.0:1788845021.900086:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641438464 00010000:00010000:3.0:1788845021.900122:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 14 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.900211:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 265 00000100:00100000:3.0:1788845021.900221:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641438464:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845021.900238:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845021.900271:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 14 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.900289:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 14 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.900314:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db59df10/0xb2187301287ac1b5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1568768->1626111) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7b9f expref: 14 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.900331:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 10) 00010000:00010000:3.0:1788845021.900335:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 10) 00010000:00010000:3.0:1788845021.900343:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.900353:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.900385:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.900413:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660199168:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845021.900488:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845021.904228:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660199168:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845021.904252:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845021.904336:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641438464:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4115us (4343us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.904346:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 265 00000100:00100000:2.0:1788845021.907754:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788845021.907787:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660199296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845021.907851:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845021.908001:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660199296 00000100:00100000:0.0:1788845021.908008:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660199296 00000100:00100000:0.0:1788845021.908067:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 76 00000100:00100000:0.0:1788845021.908077:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660199296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845021.908113:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845021.908160:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660199296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.908167:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 76 00000100:00100000:0.0:1788845021.908216:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660199296:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845021.910362:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.910448:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641438592 00000100:00100000:1.0:1788845021.910469:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641438592 00002000:00100000:1.0:1788845021.910509:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f776680 x1875739641438592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 520/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.910520:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f776680 with x1875739641438592 00000100:00100000:1.0:1788845021.910651:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 421 00000100:00100000:1.0:1788845021.910664:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641438592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.926269:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641438592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 15587us (15893us total) trans 0 rc 864256/864256 00000100:00100000:1.0:1788845021.926284:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 421 00002000:00100000:1.0:1788845021.926296:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f776680 x1875739641438592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 520/440 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 864256/864256 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.926307:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f776680 with x1875739641438592 00000100:00100000:1.0:1788845021.932552:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.932652:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641438720 00000100:00100000:3.0:1788845021.932663:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641438720 00000100:00100000:3.0:1788845021.932724:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 730 00000100:00100000:3.0:1788845021.932735:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641438720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845021.932757:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845021.932760:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845021.932809:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ac241 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c1d expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.932828:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.932833:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845021.932847:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c55 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.932871:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.932933:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.932980:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.932996:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 16 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845021.933008:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660199424:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845021.933100:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845021.937799:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.937940:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660199424:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845021.937974:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.937993:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845021.937996:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845021.938001:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.938011:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:3.0:1788845021.938084:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641438720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5350us (5542us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845021.938093:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 730 00000100:00100000:2.0:1788845021.938228:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641438848 00000100:00100000:2.0:1788845021.938237:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641438848 00010000:00010000:2.0:1788845021.938269:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 17 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.938303:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 266 00000100:00100000:2.0:1788845021.938311:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641438848:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845021.938324:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845021.938350:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 17 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.938367:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 17 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.938390:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac256 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c32 expref: 17 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.938403:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845021.938407:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.938415:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.938424:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.938474:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.938488:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660199552:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845021.938541:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845021.941238:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660199552:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845021.941261:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845021.941305:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641438848:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2993us (3517us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.941315:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 266 00000100:00100000:1.0:1788845021.943565:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.943652:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641438976 00000100:00100000:3.0:1788845021.943662:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641438976 00000100:00100000:3.0:1788845021.943731:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 422 00000100:00100000:3.0:1788845021.943740:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641438976:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845021.944030:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031310 -> 1031310 00000004:00010000:3.0:1788845021.944044:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.944046:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.944048:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845021.944051:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845021.944117:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641438976:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 378us (571us total) trans 4294967596 rc 0/0 00000100:00100000:3.0:1788845021.944125:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 422 00000100:00100000:0.0:1788845021.948394:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.948538:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641439104 00000100:00100000:1.0:1788845021.948548:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641439104 00000100:00100000:1.0:1788845021.948616:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 472 00000100:00100000:1.0:1788845021.948627:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641439104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845021.948646:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845021.948665:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c5c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.948697:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.948707:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 10) 00010000:00010000:1.0:1788845021.948722:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:1.0:1788845021.948729:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1236992->1302527] (req 1236992->1302527) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 14 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.948764:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845021.948782:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 9 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845021.948796:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660199680:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845021.948878:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.952203:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.952283:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641439232 00000100:00100000:3.0:1788845021.952293:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641439232 00002000:00100000:3.0:1788845021.952347:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1544192->1577926) req@ffff94a03d5de680 x1875739641439232/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.952357:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5de680 with x1875739641439232 ext(1544192->1577926) 00010000:00010000:3.0:1788845021.952366:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 10 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.952388:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 10 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845021.952395:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03d5de680 00000100:00100000:3.0:1788845021.952431:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 11 00000100:00100000:3.0:1788845021.952441:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641439232:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845021.953218:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660199680:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845021.953251:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1236992->1302527] (req 1236992->1302527) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 14 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845021.953284:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a800, rc 0) 00000100:00100000:0.0:1788845021.953335:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641439104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4710us (4951us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845021.953343:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 472 00000100:00100000:3.0:1788845021.958170:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641439232:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5721us (5966us total) trans 4294967384 rc 0/0 00000100:00100000:3.0:1788845021.958196:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 11 00002000:00100000:3.0:1788845021.958210:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1544192->1577926) req@ffff94a03d5de680 x1875739641439232/t4294967384(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:127/0 lens 488/448 e 0 to 0 dl 1788845032 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.958219:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5de680 with x1875739641439232 ext(1544192->1577926) 00010000:00010000:3.0:1788845021.958230:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 12 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845021.958251:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 12 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845021.958258:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03d5de680 00000100:00100000:1.0:1788845021.961359:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.961527:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641439360 00000100:00100000:2.0:1788845021.961536:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641439360 00010000:00010000:2.0:1788845021.961582:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 12 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.961618:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 267 00000100:00100000:2.0:1788845021.961628:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641439360:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845021.961641:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1788845021.961658:0:5825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1061925 -> 1577927 00010000:00010000:2.0:1788845021.961664:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 12 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.961681:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 12 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.961722:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac279 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1581055) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c4e expref: 12 pid: 6684 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.961739:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:2.0:1788845021.961743:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:2.0:1788845021.961751:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.961760:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845021.961792:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845021.961805:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660199808:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845021.961879:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845021.964877:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.964998:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641439488 00000100:00100000:3.0:1788845021.965008:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641439488 00002000:00100000:3.0:1788845021.965067:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1236992->1241087) req@ffff94a010482a00 x1875739641439488/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.965078:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010482a00 with x1875739641439488 ext(1236992->1241087) 00002000:00010000:3.0:1788845021.965084:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010482a00 00000100:00100000:3.0:1788845021.965108:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 260 00000100:00100000:3.0:1788845021.965118:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641439488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.965351:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641439488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (482us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845021.965358:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 260 00002000:00100000:3.0:1788845021.965364:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1236992->1241087) req@ffff94a010482a00 x1875739641439488/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.965390:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010482a00 with x1875739641439488 ext(1236992->1241087) 00002000:00010000:3.0:1788845021.965394:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010482a00 00000100:00100000:2.0:1788845021.965859:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660199808:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845021.966032:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845021.966078:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641439360:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4451us (4729us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.966087:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 267 00000100:00100000:1.0:1788845021.968995:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.969088:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641439616 00000100:00100000:3.0:1788845021.969098:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641439616 00002000:00100000:3.0:1788845021.969141:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1241088->1245183) req@ffff94a010481500 x1875739641439616/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.969152:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010481500 with x1875739641439616 ext(1241088->1245183) 00002000:00010000:3.0:1788845021.969157:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010481500 00000100:00100000:3.0:1788845021.969185:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 261 00000100:00100000:3.0:1788845021.969193:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641439616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.969415:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641439616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 221us (428us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845021.969423:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 261 00002000:00100000:3.0:1788845021.969430:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1241088->1245183) req@ffff94a010481500 x1875739641439616/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.969438:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010481500 with x1875739641439616 ext(1241088->1245183) 00002000:00010000:3.0:1788845021.969442:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010481500 00000100:00100000:1.0:1788845021.972563:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.972668:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641439744 00000100:00100000:0.0:1788845021.972678:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641439744 00002000:00100000:0.0:1788845021.972732:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff94a010482300 x1875739641439744/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845021.972743:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010482300 with x1875739641439744 ext(1245184->1249279) 00002000:00010000:0.0:1788845021.972749:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010482300 00000100:00100000:0.0:1788845021.972783:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 262 00000100:00100000:0.0:1788845021.972794:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641439744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845021.973093:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641439744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 300us (540us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845021.973101:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 262 00002000:00100000:0.0:1788845021.973109:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff94a010482300 x1875739641439744/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845021.973117:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010482300 with x1875739641439744 ext(1245184->1249279) 00002000:00010000:0.0:1788845021.973121:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010482300 00000100:00100000:0.0:1788845021.976386:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.976489:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845021.976590:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641440000 00000100:00100000:2.0:1788845021.976601:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641440000 00000100:00100000:2.0:1788845021.976668:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 277 00000100:00100000:2.0:1788845021.976693:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2370:x1875739641440000:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845021.976773:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2370:x1875739641440000:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 82us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845021.976780:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 277 00000100:00100000:1.0:1788845021.977546:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641439872 00000100:00100000:1.0:1788845021.977598:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641439872 00002000:00100000:1.0:1788845021.977638:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1253375) req@ffff94a035444380 x1875739641439872/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.977648:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035444380 with x1875739641439872 ext(1249280->1253375) 00002000:00010000:1.0:1788845021.977654:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035444380 00000100:00100000:1.0:1788845021.977671:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 263 00000100:00100000:1.0:1788845021.977677:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035444380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641439872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.978059:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035444380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641439872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 380us (1684us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845021.978079:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 263 00002000:00100000:1.0:1788845021.978086:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1253375) req@ffff94a035444380 x1875739641439872/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.978093:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035444380 with x1875739641439872 ext(1249280->1253375) 00002000:00010000:1.0:1788845021.978097:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035444380 00000100:00100000:0.0:1788845021.981256:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.981346:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641440128 00000100:00100000:3.0:1788845021.981358:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641440128 00002000:00100000:3.0:1788845021.981409:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1257471) req@ffff94a035444a80 x1875739641440128/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.981418:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035444a80 with x1875739641440128 ext(1253376->1257471) 00002000:00010000:3.0:1788845021.981424:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035444a80 00000100:00100000:3.0:1788845021.981469:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 264 00000100:00100000:3.0:1788845021.981477:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035444a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641440128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.981719:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035444a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641440128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 240us (473us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845021.981727:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 264 00002000:00100000:3.0:1788845021.981733:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1257471) req@ffff94a035444a80 x1875739641440128/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.981742:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035444a80 with x1875739641440128 ext(1253376->1257471) 00002000:00010000:3.0:1788845021.981746:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035444a80 00000100:00100000:2.0:1788845021.984495:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.984594:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641440256 00000100:00100000:0.0:1788845021.984617:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641440256 00002000:00100000:0.0:1788845021.984663:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1257472->1261567) req@ffff94a0359d4a80 x1875739641440256/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845021.984673:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359d4a80 with x1875739641440256 ext(1257472->1261567) 00002000:00010000:0.0:1788845021.984678:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359d4a80 00000100:00100000:0.0:1788845021.984709:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 265 00000100:00100000:0.0:1788845021.984721:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641440256:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845021.984987:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641440256:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (501us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845021.984994:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 265 00002000:00100000:0.0:1788845021.985002:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1257472->1261567) req@ffff94a0359d4a80 x1875739641440256/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845021.985010:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359d4a80 with x1875739641440256 ext(1257472->1261567) 00002000:00010000:0.0:1788845021.985013:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359d4a80 00000100:00100000:0.0:1788845021.988094:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.988189:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641440384 00000100:00100000:1.0:1788845021.988199:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641440384 00002000:00100000:1.0:1788845021.988247:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1261568->1265663) req@ffff94a035446680 x1875739641440384/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.988258:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035446680 with x1875739641440384 ext(1261568->1265663) 00002000:00010000:1.0:1788845021.988263:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035446680 00000100:00100000:1.0:1788845021.988296:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 266 00000100:00100000:1.0:1788845021.988307:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035446680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641440384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.988537:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641440384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (451us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845021.988545:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 266 00002000:00100000:1.0:1788845021.989155:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1261568->1265663) req@ffff94a035446680 x1875739641440384/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.989165:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035446680 with x1875739641440384 ext(1261568->1265663) 00002000:00010000:1.0:1788845021.989169:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035446680 00000100:00100000:1.0:1788845021.991650:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845021.991742:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641440512 00000100:00100000:3.0:1788845021.991753:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641440512 00002000:00100000:3.0:1788845021.991801:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1265664->1269759) req@ffff94a010483100 x1875739641440512/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.991813:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010483100 with x1875739641440512 ext(1265664->1269759) 00002000:00010000:3.0:1788845021.991818:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010483100 00000100:00100000:3.0:1788845021.991853:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 267 00000100:00100000:3.0:1788845021.991863:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641440512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845021.992143:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641440512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (501us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845021.992150:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 267 00002000:00100000:3.0:1788845021.992157:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1265664->1269759) req@ffff94a010483100 x1875739641440512/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845021.992165:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010483100 with x1875739641440512 ext(1265664->1269759) 00002000:00010000:3.0:1788845021.992183:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010483100 00000100:00100000:2.0:1788845021.994961:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845021.995107:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641440640 00000100:00100000:1.0:1788845021.995117:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641440640 00002000:00100000:1.0:1788845021.995177:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1269760->1273855) req@ffff94a0359d5c00 x1875739641440640/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.995188:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359d5c00 with x1875739641440640 ext(1269760->1273855) 00002000:00010000:1.0:1788845021.995194:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359d5c00 00000100:00100000:1.0:1788845021.995226:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 268 00000100:00100000:1.0:1788845021.995236:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641440640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845021.995475:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641440640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 239us (521us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845021.995483:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 268 00002000:00100000:1.0:1788845021.995491:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1269760->1273855) req@ffff94a0359d5c00 x1875739641440640/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845021.995498:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359d5c00 with x1875739641440640 ext(1269760->1273855) 00002000:00010000:1.0:1788845021.995502:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359d5c00 00000100:00100000:1.0:1788845021.998487:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845021.998599:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641440768 00000100:00100000:0.0:1788845021.998610:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641440768 00002000:00100000:0.0:1788845021.998660:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a010483800 x1875739641440768/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/0 e 0 to 0 dl 1788845032 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845021.998673:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010483800 with x1875739641440768 ext(1273856->1277951) 00002000:00010000:0.0:1788845021.998679:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010483800 00000100:00100000:0.0:1788845021.998713:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 269 00000100:00100000:0.0:1788845021.998723:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641440768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845021.999003:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641440768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (524us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845021.999024:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 269 00002000:00100000:0.0:1788845021.999031:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a010483800 x1875739641440768/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:127/0 lens 488/4536 e 0 to 0 dl 1788845032 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845021.999050:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010483800 with x1875739641440768 ext(1273856->1277951) 00002000:00010000:0.0:1788845021.999054:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010483800 00000100:00100000:0.0:1788845022.002559:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.002659:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641440896 00000100:00100000:3.0:1788845022.002669:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641440896 00002000:00100000:3.0:1788845022.002722:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1277952->1282047) req@ffff94a035446300 x1875739641440896/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.002733:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035446300 with x1875739641440896 ext(1277952->1282047) 00002000:00010000:3.0:1788845022.002738:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035446300 00000100:00100000:3.0:1788845022.002769:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 270 00000100:00100000:3.0:1788845022.002779:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035446300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641440896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.003532:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035446300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641440896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 750us (978us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845022.003540:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 270 00002000:00100000:0.0:1788845022.003548:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1277952->1282047) req@ffff94a035446300 x1875739641440896/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.003566:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035446300 with x1875739641440896 ext(1277952->1282047) 00002000:00010000:0.0:1788845022.003570:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035446300 00000100:00100000:2.0:1788845022.008865:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.009287:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641441024 00000100:00100000:1.0:1788845022.009297:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641441024 00002000:00100000:1.0:1788845022.009349:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1282048->1286143) req@ffff94a0359d4380 x1875739641441024/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.009508:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359d4380 with x1875739641441024 ext(1282048->1286143) 00002000:00010000:1.0:1788845022.009514:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359d4380 00000100:00100000:1.0:1788845022.009648:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 271 00000100:00100000:1.0:1788845022.009659:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641441024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.010292:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641441024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 632us (1433us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845022.010318:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 271 00002000:00100000:1.0:1788845022.010327:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1282048->1286143) req@ffff94a0359d4380 x1875739641441024/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.010337:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359d4380 with x1875739641441024 ext(1282048->1286143) 00002000:00010000:1.0:1788845022.010341:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359d4380 00000100:00100000:3.0:1788845022.013288:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.014338:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641441152 00000100:00100000:1.0:1788845022.014353:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641441152 00002000:00100000:1.0:1788845022.014411:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1286144->1290239) req@ffff94a036080000 x1875739641441152/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.014422:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036080000 with x1875739641441152 ext(1286144->1290239) 00002000:00010000:1.0:1788845022.014427:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036080000 00000100:00100000:1.0:1788845022.014452:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 272 00000100:00100000:1.0:1788845022.014461:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641441152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.014722:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641441152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 260us (1443us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845022.014731:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 272 00002000:00100000:1.0:1788845022.014737:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1286144->1290239) req@ffff94a036080000 x1875739641441152/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.014746:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036080000 with x1875739641441152 ext(1286144->1290239) 00002000:00010000:1.0:1788845022.014750:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036080000 00000100:00100000:2.0:1788845022.018284:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.018739:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641441280 00000100:00100000:1.0:1788845022.018750:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641441280 00002000:00100000:1.0:1788845022.018801:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1290240->1294335) req@ffff94a03f657b80 x1875739641441280/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.018832:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f657b80 with x1875739641441280 ext(1290240->1294335) 00002000:00010000:1.0:1788845022.018838:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f657b80 00000100:00100000:1.0:1788845022.018868:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 273 00000100:00100000:1.0:1788845022.018925:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641441280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.019269:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641441280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 392us (994us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845022.019277:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 273 00002000:00100000:1.0:1788845022.019285:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1290240->1294335) req@ffff94a03f657b80 x1875739641441280/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.019293:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f657b80 with x1875739641441280 ext(1290240->1294335) 00002000:00010000:1.0:1788845022.019297:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f657b80 00000100:00100000:2.0:1788845022.023425:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.023520:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641441408 00000100:00100000:0.0:1788845022.023531:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641441408 00002000:00100000:0.0:1788845022.023650:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1294336->1298431) req@ffff94a03f657100 x1875739641441408/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.023661:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f657100 with x1875739641441408 ext(1294336->1298431) 00002000:00010000:0.0:1788845022.023667:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f657100 00000100:00100000:0.0:1788845022.028949:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 274 00000100:00100000:0.0:1788845022.028975:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641441408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.029295:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641441408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 331us (5879us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845022.029323:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 274 00002000:00100000:0.0:1788845022.029334:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1294336->1298431) req@ffff94a03f657100 x1875739641441408/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.029344:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f657100 with x1875739641441408 ext(1294336->1298431) 00002000:00010000:0.0:1788845022.029350:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f657100 00000100:00100000:2.0:1788845022.035442:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.035539:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641441536 00000100:00100000:1.0:1788845022.035549:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641441536 00002000:00100000:1.0:1788845022.035602:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1298432->1302527) req@ffff94a03f655f80 x1875739641441536/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.035612:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f655f80 with x1875739641441536 ext(1298432->1302527) 00002000:00010000:1.0:1788845022.035618:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f655f80 00000100:00100000:1.0:1788845022.036282:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 275 00000100:00100000:1.0:1788845022.036293:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641441536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.036500:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641441536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 209us (1066us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845022.036507:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 275 00002000:00100000:1.0:1788845022.036529:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1298432->1302527) req@ffff94a03f655f80 x1875739641441536/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.036537:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f655f80 with x1875739641441536 ext(1298432->1302527) 00002000:00010000:1.0:1788845022.036541:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f655f80 00000100:00100000:1.0:1788845022.046745:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.047028:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641441664 00000100:00100000:2.0:1788845022.047048:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641441664 00000100:00100000:2.0:1788845022.047118:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 473 00000100:00100000:2.0:1788845022.047129:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641441664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.047148:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.047165:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c63 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.047191:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:2.0:1788845022.047195:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:2.0:1788845022.047205:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.047222:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.047259:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845022.047328:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641441664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 200us (591us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.047334:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 473 00000100:00100000:3.0:1788845022.053540:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.053657:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641441920 00000100:00100000:0.0:1788845022.053668:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641441920 00000100:00100000:0.0:1788845022.053741:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 731 00000100:00100000:0.0:1788845022.053752:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641441920:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845022.053809:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845022.053836:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac272 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c47 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.053854:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845022.053858:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845022.054313:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.054328:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.054337:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.054370:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.054405:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 11 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.054418:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035445500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660199936:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845022.054593:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.062873:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.066103:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035445500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660199936:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845022.066140:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.069316:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641442048 00000100:00100000:2.0:1788845022.069335:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641442048 00010000:00010000:2.0:1788845022.069390:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 12 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.069508:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 268 00000100:00100000:2.0:1788845022.069519:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035447480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641442048:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845022.069534:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845022.069541:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 12 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.069570:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 12 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.069594:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ac202 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7bde expref: 12 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.069612:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.069617:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.069645:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.069653:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.069695:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845022.069763:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035447480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641442048:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 245us (6899us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.069771:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 268 00010000:00010000:0.0:1788845022.069828:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.069837:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845022.070984:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967597 00000004:00010000:0.0:1788845022.071060:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031310 -> 1031310 00000004:00010000:0.0:1788845022.071064:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845021 -> 1788845022 00000004:00010000:0.0:1788845022.071066:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:0.0:1788845022.071068:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845021 -> 1788845022 00000004:00010000:0.0:1788845022.071070:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:0.0:1788845022.071078:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000c35654b reply state 00000000546df1d6 transno 4294967598 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.071096:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845022.071212:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641441920:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 17459us (17680us total) trans 4294967598 rc 0/0 00000100:00100000:0.0:1788845022.071220:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 731 00000020:00080000:0.0:1788845022.071225:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845022.076112:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.076128:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.076137:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.076143:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008cba400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845022.076147:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.076164:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.076182:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.076185:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845022.076190:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.076195:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac295 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.079786:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.079913:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641442176 00000100:00100000:2.0:1788845022.079924:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641442176 00000100:00100000:2.0:1788845022.079979:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 732 00000100:00100000:2.0:1788845022.079988:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641442176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.080022:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.080051:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c6a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.080081:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.080100:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.080131:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.080147:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 10 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.080469:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660200064:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845022.080548:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.084525:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660200064:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845022.084576:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.084596:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.084600:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845022.084605:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.084615:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:2.0:1788845022.084684:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641442176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4697us (4908us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.084694:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 732 00000100:00100000:1.0:1788845022.093530:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.093785:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641442304 00000100:00100000:3.0:1788845022.093798:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641442304 00010000:00010000:3.0:1788845022.093838:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 11 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.093935:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 269 00000100:00100000:3.0:1788845022.093947:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641442304:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845022.093963:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845022.093992:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 11 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.094049:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 11 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.094074:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac280 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c55 expref: 11 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.094099:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.094106:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.094117:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.094127:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.094182:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.094197:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660200192:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845022.094281:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845022.099589:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660200192:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845022.099618:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845022.100433:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641442304:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6486us (6909us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.100463:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 269 00000100:00100000:3.0:1788845022.101407:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.101593:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641442432 00000100:00100000:2.0:1788845022.101602:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641442432 00000100:00100000:2.0:1788845022.101656:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 474 00000100:00100000:2.0:1788845022.101666:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641442432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.101685:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.101703:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c71 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.101734:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 15 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.101753:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.101761:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:2.0:1788845022.101766:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:2.0:1788845022.101771:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 15 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.101806:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.101825:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 11 pid: 7983 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.101839:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660200320:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845022.101949:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 15 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.101959:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 15 pid: 6685 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.101969:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660200448:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845022.101992:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.107651:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.107694:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.107752:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641442560 00000100:00100000:2.0:1788845022.107761:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641442560 00010000:00010000:2.0:1788845022.107803:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 12 pid: 7983 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.107837:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641442688 00000100:00100000:2.0:1788845022.107841:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641442688 00010000:00010000:2.0:1788845022.107855:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 16 pid: 6685 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.107897:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 270 00000100:00100000:2.0:1788845022.107914:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641442560:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845022.107930:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845022.107951:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 12 pid: 7983 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.107968:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 12 pid: 7983 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.107992:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845022.108102:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 271 00000100:00100000:0.0:1788845022.108112:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008915850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641442688:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845022.108120:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845022.108131:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 16 pid: 6685 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.108142:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 16 pid: 6685 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.108166:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:0.0:1788845022.108171:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 10) 00010000:00010000:0.0:1788845022.108179:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.108189:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.108226:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.108241:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035445500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660200576:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845022.109642:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641442560:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 1729us (1995us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.109656:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 270 00010000:00010000:2.0:1788845022.109734:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287ac28e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c63 expref: 11 pid: 7983 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.109752:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660200320:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845022.109787:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ac287 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1236992->1302527) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c5c expref: 16 pid: 6685 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.109797:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660200448:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845022.109828:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 15 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.109841:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864000, rc 0) 00000100:00100000:0.0:1788845022.110321:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.114304:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641442432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12626us (12887us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.114317:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 474 00000100:00100000:0.0:1788845022.117788:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035445500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660200576:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845022.117820:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845022.118346:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008915850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641442688:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10235us (10649us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.118372:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 271 00000100:00100000:2.0:1788845022.121739:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.121852:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.122299:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641442816 00000100:00100000:1.0:1788845022.122310:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641442816 00002000:00010000:1.0:1788845022.122352:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845022.122357:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657800 with x1875739641442816 00000100:00100000:1.0:1788845022.122397:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 423 00000100:00100000:1.0:1788845022.122407:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641442816:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845022.122787:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031310 -> 49754 00000004:00010000:1.0:1788845022.122791:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00000004:00010000:1.0:1788845022.122794:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:1.0:1788845022.122796:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00000004:00010000:1.0:1788845022.122799:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:1.0:1788845022.122873:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641442816:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 467us (1019us total) trans 4294967599 rc 0/0 00000100:00100000:1.0:1788845022.123106:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 423 00002000:00010000:1.0:1788845022.123112:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845022.123115:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657800 with x1875739641442816 00000100:00100000:3.0:1788845022.125127:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641442944 00000100:00100000:3.0:1788845022.125140:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641442944 00002000:00010000:2.0:1788845022.126295:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845022.126304:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f654a80 with x1875739641442944 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845022.126316:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f654a80. 00000100:00100000:2.0:1788845022.126414:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 276 00000100:00100000:2.0:1788845022.126425:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641442944:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845022.126771:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1577927 -> 0 00002000:00010000:2.0:1788845022.126776:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845021 -> 1788845022 00002000:00010000:2.0:1788845022.126778:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00002000:00010000:2.0:1788845022.126781:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845021 -> 1788845022 00002000:00010000:2.0:1788845022.126783:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 341 -> 341 00000100:00100000:2.0:1788845022.126856:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641442944:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 432us (5126us total) trans 4294967385 rc 0/0 00000100:00100000:2.0:1788845022.126876:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 276 00002000:00010000:2.0:1788845022.126880:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845022.126904:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f654a80 with x1875739641442944 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845022.126919:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f654a80. 00000100:00100000:3.0:1788845022.130456:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.130708:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641443072 00000100:00100000:1.0:1788845022.130718:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641443072 00000100:00100000:1.0:1788845022.130781:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 733 00000100:00100000:1.0:1788845022.130791:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641443072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.130815:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.130834:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ac2aa lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c78 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.130955:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac2b1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.130982:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac2b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.130993:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac2b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845022.131017:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 49754 00000004:00200000:1.0:1788845022.131021:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845022.131048:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac2b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.131075:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ac2aa lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c78 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.131103:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac2b1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c78 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.131113:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.131115:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845022.131118:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b600, rc 0) 00000100:00100000:1.0:1788845022.131613:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641443072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 821us (1165us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.131622:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 733 00000100:00100000:3.0:1788845022.136379:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.136559:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641443328 00000100:00100000:2.0:1788845022.136569:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641443328 00000100:00100000:2.0:1788845022.136629:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 734 00000100:00100000:2.0:1788845022.136639:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641443328:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845022.136693:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845022.136722:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ac2b1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c78 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.136741:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.136745:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845022.136975:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.136989:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.136999:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845022.137070:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967600 00000004:00010000:2.0:1788845022.137094:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 49754 -> 49754 00000004:00010000:2.0:1788845022.137098:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00000004:00010000:2.0:1788845022.137101:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845022.137103:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00000004:00010000:2.0:1788845022.137117:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:2.0:1788845022.137123:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e9d39796 reply state 00000000d48262bf transno 4294967601 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.137132:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845022.137227:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641443328:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 589us (856us total) trans 4294967601 rc 0/0 00000100:00100000:2.0:1788845022.137234:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 734 00000020:00080000:2.0:1788845022.137239:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845022.141685:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.141702:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.141710:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.141717:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845022.141722:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.141732:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.141752:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845022.141755:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845022.141761:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.141767:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ac2b8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.143142:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.144054:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641443456 00000100:00100000:2.0:1788845022.144065:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641443456 00002000:00010000:2.0:1788845022.144115:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1239042->18446744073709551615) 00002000:00100000:2.0:1788845022.144120:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc5180 with x1875739641443456 ext(1239042->18446744073709551615) 00002000:00010000:2.0:1788845022.144133:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc5180. 00000100:00100000:2.0:1788845022.144321:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 277 00000100:00100000:2.0:1788845022.144335:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641443456:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845022.144564:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1239042 00002000:00010000:2.0:1788845022.144568:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00002000:00010000:2.0:1788845022.144570:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00002000:00010000:2.0:1788845022.144572:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00002000:00010000:2.0:1788845022.144574:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 341 -> 341 00000100:00100000:2.0:1788845022.144644:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641443456:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 312us (1510us total) trans 4294967386 rc 0/0 00000100:00100000:2.0:1788845022.144651:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 277 00002000:00010000:2.0:1788845022.144655:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1239042->18446744073709551615) 00002000:00100000:2.0:1788845022.144657:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc5180 with x1875739641443456 ext(1239042->18446744073709551615) 00002000:00010000:2.0:1788845022.144662:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc5180. 00000100:00100000:3.0:1788845022.156185:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.156234:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.156453:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641443584 00000100:00100000:2.0:1788845022.156465:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641443584 00002000:00100000:2.0:1788845022.156514:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1241087) req@ffff94a0359bad80 x1875739641443584/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.156528:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359bad80 with x1875739641443584 ext(1048576->1241087) 00002000:00010000:2.0:1788845022.156534:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359bad80 00000100:00100000:2.0:1788845022.156573:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 278 00000100:00100000:2.0:1788845022.156586:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641443584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.157645:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641443712 00000100:00100000:0.0:1788845022.157652:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641443712 00002000:00100000:0.0:1788845022.157691:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9180 x1875739641443712/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.157701:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9180 with x1875739641443712 00000100:00100000:0.0:1788845022.157742:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 424 00000100:00100000:0.0:1788845022.157751:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641443712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.165593:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641443584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8998us (9406us total) trans 0 rc 192512/192512 00000100:00100000:1.0:1788845022.165608:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 278 00002000:00100000:1.0:1788845022.165620:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1241087) req@ffff94a0359bad80 x1875739641443584/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 192512/192512 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.165642:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359bad80 with x1875739641443584 ext(1048576->1241087) 00002000:00010000:1.0:1788845022.165647:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359bad80 00000100:00100000:0.0:1788845022.166343:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641443712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8590us (10106us total) trans 0 rc 53248/53248 00000100:00100000:0.0:1788845022.166353:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 424 00002000:00100000:0.0:1788845022.166361:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9180 x1875739641443712/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 53248/53248 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.166371:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9180 with x1875739641443712 00000100:00100000:3.0:1788845022.174571:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.174739:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641443840 00000100:00100000:0.0:1788845022.174749:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641443840 00002000:00100000:0.0:1788845022.174799:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1208320->1239041) req@ffff94a0360ece00 x1875739641443840/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.174811:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ece00 with x1875739641443840 ext(1208320->1239041) 00002000:00010000:0.0:1788845022.174817:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ece00 00000100:00100000:0.0:1788845022.174956:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 279 00000100:00100000:0.0:1788845022.174966:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641443840:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845022.179781:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641443840:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4805us (5220us total) trans 4294967387 rc 0/0 00000100:00100000:0.0:1788845022.179795:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 279 00002000:00100000:0.0:1788845022.179808:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1208320->1239041) req@ffff94a0360ece00 x1875739641443840/t4294967387(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/448 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.179819:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ece00 with x1875739641443840 ext(1208320->1239041) 00002000:00010000:0.0:1788845022.179826:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ece00 00000100:00100000:3.0:1788845022.185594:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.187209:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641443968 00000100:00100000:2.0:1788845022.187221:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641443968 00000100:00100000:2.0:1788845022.187289:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 475 00000100:00100000:2.0:1788845022.187299:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641443968:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:0.0:1788845022.223245:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641443968:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 35936us (37651us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.223274:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 475 00000001:00080000:2.0:1788845022.225170:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967387 is committed 00000001:00080000:2.0:1788845022.225200:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967379 is committed 00000001:00080000:2.0:1788845022.225222:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967384 is committed 00000100:00100000:2.0:1788845022.226639:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.226858:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641444096 00000100:00100000:3.0:1788845022.226869:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641444096 00000100:00100000:3.0:1788845022.227059:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 735 00000100:00100000:3.0:1788845022.227070:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641444096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.227091:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.227112:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2bf lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c7f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.227189:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac2c6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.227202:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac2c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.227213:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac2c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845022.227240:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1239042 00000004:00200000:3.0:1788845022.227258:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845022.227281:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac2c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.227309:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2bf lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c7f expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.227327:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac2c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c7f expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.227335:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.227338:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845022.227342:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:3.0:1788845022.227458:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641444096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 389us (827us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.227466:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 735 00000100:00100000:1.0:1788845022.231529:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.231718:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641444224 00000100:00100000:2.0:1788845022.231728:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641444224 00000100:00100000:2.0:1788845022.231816:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 736 00000100:00100000:2.0:1788845022.231827:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641444224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.231851:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.231870:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c86 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.231932:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.231963:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c86 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.231998:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.232015:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 21 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.232038:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660200704:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845022.232181:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01373aa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.234220:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660200704:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845022.234247:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c86 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.234261:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.234264:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845022.234266:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c86 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.234274:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864e00, rc 0) 00000100:00100000:2.0:1788845022.234316:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641444224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2492us (2798us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.234321:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 736 00000100:00100000:0.0:1788845022.237277:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.237380:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641444352 00000100:00100000:2.0:1788845022.237401:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641444352 00010000:00010000:2.0:1788845022.237432:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 22 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.237466:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 272 00000100:00100000:2.0:1788845022.237473:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035447480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:2369:x1875739641444352:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845022.237485:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845022.237506:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 22 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.237519:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 22 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.237537:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287ac29c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c6a expref: 22 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.237556:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.237561:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c86 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.237568:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c86 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.237575:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c86 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.237600:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c86 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.237619:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660200832:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845022.237688:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01373aa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.244314:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660200832:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845022.244342:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845022.244401:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035447480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641444352:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6926us (7128us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.244413:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 272 00000100:00100000:3.0:1788845022.244553:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.244684:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641444480 00000100:00100000:1.0:1788845022.244691:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641444480 00000100:00100000:1.0:1788845022.244740:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 476 00000100:00100000:1.0:1788845022.244749:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641444480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.244767:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.244785:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac2d4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c8d expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.244839:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660200960:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.244938:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00002000:00010000:1.0:1788845022.248326:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845021 -> 1788845022 00000100:00100000:1.0:1788845022.248359:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660200960:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845022.248397:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac2d4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c8d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.248423:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac2d4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c8d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.248440:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c4200, rc 0) 00000100:00100000:1.0:1788845022.248507:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641444480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3757us (3958us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.248534:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 476 00000100:00100000:3.0:1788845022.251954:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.252053:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641444608 00000100:00100000:0.0:1788845022.252063:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641444608 00002000:00100000:0.0:1788845022.252109:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ef100 x1875739641444608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 504/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.252120:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ef100 with x1875739641444608 00000100:00100000:0.0:1788845022.252149:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 425 00000100:00100000:0.0:1788845022.252158:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641444608:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.261140:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641444608:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8971us (9183us total) trans 0 rc 53248/53248 00000100:00100000:0.0:1788845022.261155:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 425 00002000:00100000:0.0:1788845022.261168:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ef100 x1875739641444608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 504/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 53248/53248 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.261178:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ef100 with x1875739641444608 00000100:00100000:1.0:1788845022.263516:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.263662:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641444736 00000100:00100000:3.0:1788845022.263837:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641444736 00002000:00100000:3.0:1788845022.263917:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5f80 x1875739641444736/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.263929:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5f80 with x1875739641444736 00000100:00100000:3.0:1788845022.263969:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 426 00000100:00100000:3.0:1788845022.263979:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641444736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.264377:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641444736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 397us (868us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.264403:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 426 00002000:00100000:3.0:1788845022.264411:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5f80 x1875739641444736/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.264419:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5f80 with x1875739641444736 00000100:00100000:1.0:1788845022.266693:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.266783:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641444864 00000100:00100000:2.0:1788845022.266792:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641444864 00002000:00100000:2.0:1788845022.266833:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5500 x1875739641444864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.266844:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5500 with x1875739641444864 00000100:00100000:2.0:1788845022.267001:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 427 00000100:00100000:2.0:1788845022.267015:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641444864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845022.267246:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641444864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (560us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.269748:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.269855:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641444992 00000100:00100000:0.0:1788845022.269864:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641444992 00002000:00100000:0.0:1788845022.269951:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ed880 x1875739641444992/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.269964:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ed880 with x1875739641444992 00000100:00100000:2.0:1788845022.270219:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 427 00002000:00100000:2.0:1788845022.270229:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5500 x1875739641444864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.270237:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5500 with x1875739641444864 00000100:00100000:0.0:1788845022.270329:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 428 00000100:00100000:0.0:1788845022.270341:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641444992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.270724:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641444992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 384us (985us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.270750:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 428 00002000:00100000:0.0:1788845022.270757:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ed880 x1875739641444992/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.270764:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ed880 with x1875739641444992 00000100:00100000:0.0:1788845022.273567:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.273670:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641445120 00000100:00100000:1.0:1788845022.273680:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641445120 00002000:00100000:1.0:1788845022.273727:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035445500 x1875739641445120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.273738:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035445500 with x1875739641445120 00000100:00100000:1.0:1788845022.273775:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 429 00000100:00100000:1.0:1788845022.273785:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641445120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.274083:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641445120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 298us (524us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.274090:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 429 00002000:00100000:1.0:1788845022.274097:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035445500 x1875739641445120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.274106:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035445500 with x1875739641445120 00000100:00100000:2.0:1788845022.276650:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.276738:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641445248 00000100:00100000:3.0:1788845022.276747:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641445248 00002000:00100000:3.0:1788845022.276792:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657480 x1875739641445248/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.276803:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657480 with x1875739641445248 00000100:00100000:3.0:1788845022.276841:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 430 00000100:00100000:3.0:1788845022.276851:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641445248:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.277130:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641445248:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 279us (491us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.277138:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 430 00002000:00100000:3.0:1788845022.277145:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657480 x1875739641445248/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.277153:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657480 with x1875739641445248 00000100:00100000:3.0:1788845022.279666:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.279758:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641445376 00000100:00100000:1.0:1788845022.279767:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641445376 00002000:00100000:1.0:1788845022.279810:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ec380 x1875739641445376/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.279821:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ec380 with x1875739641445376 00000100:00100000:1.0:1788845022.279853:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 431 00000100:00100000:1.0:1788845022.279862:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641445376:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.280294:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641445376:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 430us (634us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.280319:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 431 00002000:00100000:1.0:1788845022.280327:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ec380 x1875739641445376/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.280335:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ec380 with x1875739641445376 00000100:00100000:1.0:1788845022.282935:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.283702:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641445504 00000100:00100000:0.0:1788845022.283711:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641445504 00002000:00100000:0.0:1788845022.283754:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6a00 x1875739641445504/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.283762:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6a00 with x1875739641445504 00000100:00100000:0.0:1788845022.283793:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 432 00000100:00100000:0.0:1788845022.283803:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641445504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.284096:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641445504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (1213us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.284104:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 432 00002000:00100000:0.0:1788845022.284116:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6a00 x1875739641445504/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.284124:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6a00 with x1875739641445504 00000100:00100000:2.0:1788845022.287134:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.287379:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641445632 00000100:00100000:1.0:1788845022.287390:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641445632 00002000:00100000:1.0:1788845022.287440:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654000 x1875739641445632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.287451:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654000 with x1875739641445632 00000100:00100000:1.0:1788845022.287489:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 433 00000100:00100000:1.0:1788845022.287499:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641445632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.288322:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641445632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 820us (1192us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.288331:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 433 00002000:00100000:1.0:1788845022.288339:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654000 x1875739641445632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.288348:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654000 with x1875739641445632 00000100:00100000:3.0:1788845022.296179:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.296293:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641445760 00000100:00100000:2.0:1788845022.296313:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641445760 00002000:00100000:2.0:1788845022.296355:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0e00 x1875739641445760/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.296364:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0e00 with x1875739641445760 00000100:00100000:2.0:1788845022.296444:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 434 00000100:00100000:2.0:1788845022.296451:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641445760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845022.296665:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641445760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 214us (497us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.296670:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 434 00002000:00100000:2.0:1788845022.296676:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0e00 x1875739641445760/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.296684:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0e00 with x1875739641445760 00000100:00100000:0.0:1788845022.300665:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.300910:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641445888 00000100:00100000:1.0:1788845022.300919:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641445888 00002000:00100000:1.0:1788845022.300965:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ce00 x1875739641445888/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.300975:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ce00 with x1875739641445888 00000100:00100000:1.0:1788845022.301026:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 435 00000100:00100000:1.0:1788845022.301067:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641445888:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.301283:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641445888:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 218us (626us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.301288:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 435 00002000:00100000:1.0:1788845022.301292:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ce00 x1875739641445888/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.301298:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ce00 with x1875739641445888 00000100:00100000:3.0:1788845022.303738:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.304159:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641446016 00000100:00100000:0.0:1788845022.304170:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641446016 00002000:00100000:0.0:1788845022.304220:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1f80 x1875739641446016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.304230:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1f80 with x1875739641446016 00000100:00100000:0.0:1788845022.304283:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 436 00000100:00100000:0.0:1788845022.304292:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641446016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.304546:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641446016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (818us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.304554:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 436 00002000:00100000:0.0:1788845022.304560:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1f80 x1875739641446016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.304568:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1f80 with x1875739641446016 00000100:00100000:3.0:1788845022.307138:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.307542:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641446144 00000100:00100000:2.0:1788845022.307621:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641446144 00002000:00100000:2.0:1788845022.307669:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb3100 x1875739641446144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.307681:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb3100 with x1875739641446144 00000100:00100000:2.0:1788845022.307770:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 437 00000100:00100000:2.0:1788845022.307781:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641446144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845022.308105:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641446144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 324us (977us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.308114:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 437 00002000:00100000:2.0:1788845022.308122:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb3100 x1875739641446144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.308130:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb3100 with x1875739641446144 00000100:00100000:3.0:1788845022.310797:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.311663:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641446272 00000100:00100000:1.0:1788845022.311675:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641446272 00002000:00100000:1.0:1788845022.311734:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1500 x1875739641446272/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.311747:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1500 with x1875739641446272 00000100:00100000:1.0:1788845022.311784:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 438 00000100:00100000:1.0:1788845022.311795:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641446272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.312118:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641446272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 323us (1329us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.312126:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 438 00002000:00100000:1.0:1788845022.312134:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1500 x1875739641446272/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.312141:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1500 with x1875739641446272 00000100:00100000:3.0:1788845022.316311:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.316499:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641446400 00000100:00100000:1.0:1788845022.316509:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641446400 00000100:00100000:1.0:1788845022.316607:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 737 00000100:00100000:1.0:1788845022.316618:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641446400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.316646:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.316668:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c94 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.316705:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845022.316719:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 49754 00010000:00010000:1.0:1788845022.316727:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.316741:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.316744:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.316748:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c4200, rc 0) 00000100:00100000:1.0:1788845022.316810:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:2367:x1875739641446400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 194us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.316817:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 737 00000100:00100000:2.0:1788845022.320476:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.320667:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641446528 00000100:00100000:3.0:1788845022.320677:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641446528 00000100:00100000:3.0:1788845022.320738:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 738 00000100:00100000:3.0:1788845022.320749:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641446528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.320774:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.320794:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac2e2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7c9b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.321116:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.321132:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.321143:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845022.321171:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1239042 00000004:00200000:3.0:1788845022.321175:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845022.321191:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.321215:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac2e2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.321232:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.321240:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.321243:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845022.321258:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:3.0:1788845022.321322:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641446528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 574us (857us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.321329:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 738 00000100:00100000:0.0:1788845022.325320:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.325435:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641446656 00000100:00100000:3.0:1788845022.325445:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641446656 00000100:00100000:3.0:1788845022.325543:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 477 00000100:00100000:3.0:1788845022.325565:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641446656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.325585:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.325605:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ac2f0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ca2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.325673:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660201088:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845022.325764:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.328350:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660201088:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845022.328390:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ac2f0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ca2 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.328414:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ac2f0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ca2 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.328427:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845022.328483:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641446656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2920us (3175us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.328503:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 477 00000100:00100000:2.0:1788845022.332764:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.332856:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641446784 00000100:00100000:3.0:1788845022.332866:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641446784 00000100:00100000:3.0:1788845022.332957:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:3.0:1788845022.332967:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641446784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.332985:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.333002:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ac2f7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ca9 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.333196:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660201216:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845022.333290:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.336235:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660201216:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845022.336301:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ac2f7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ca9 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.336331:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ac2f7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ca9 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.336349:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:2.0:1788845022.336418:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641446784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3452us (3664us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845022.336427:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:2.0:1788845022.341609:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.341703:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641446912 00000100:00100000:0.0:1788845022.341713:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641446912 00000100:00100000:0.0:1788845022.341781:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 479 00000100:00100000:0.0:1788845022.341792:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641446912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.341811:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.341829:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac2fe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cb0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.341928:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660201344:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845022.342017:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.346129:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660201344:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845022.346173:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac2fe lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cb0 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.346200:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac2fe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cb0 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.346237:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbbc00, rc 0) 00000100:00100000:0.0:1788845022.346739:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641446912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4947us (5138us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845022.346748:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 479 00000100:00100000:3.0:1788845022.350686:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.351085:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641447040 00000100:00100000:1.0:1788845022.351095:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641447040 00000100:00100000:1.0:1788845022.351151:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 480 00000100:00100000:1.0:1788845022.351162:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641447040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.351181:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.351199:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003a075281/0xb2187301287ac305 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cb7 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.351246:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660201472:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.351324:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.353395:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660201472:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845022.353432:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003a075281/0xb2187301287ac305 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cb7 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.353455:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003a075281/0xb2187301287ac305 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cb7 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.353469:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5000, rc 0) 00000100:00100000:1.0:1788845022.353521:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641447040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2361us (2845us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.353529:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 480 00000100:00100000:2.0:1788845022.357431:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.357525:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641447168 00000100:00100000:1.0:1788845022.357534:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641447168 00000100:00100000:1.0:1788845022.357613:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 481 00000100:00100000:1.0:1788845022.357623:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641447168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.357641:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.357657:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac30c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cbe expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.357717:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660201600:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.357799:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.360974:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660201600:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845022.361036:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac30c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cbe expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.361060:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac30c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cbe expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.361082:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5200, rc 0) 00000100:00100000:1.0:1788845022.361143:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641447168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3520us (3721us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.361151:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 481 00000100:00100000:0.0:1788845022.364640:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.365228:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641447296 00000100:00100000:3.0:1788845022.365240:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641447296 00000100:00100000:0.0:1788845022.365491:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 482 00000100:00100000:0.0:1788845022.365502:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641447296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.365521:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.365539:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac313 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cc5 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.365682:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f777b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660201728:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845022.365766:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.368462:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f777b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660201728:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845022.368507:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac313 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cc5 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.368532:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac313 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cc5 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.368615:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbbc00, rc 0) 00000100:00100000:0.0:1788845022.368678:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641447296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3177us (4046us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845022.368702:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 482 00000100:00100000:3.0:1788845022.371374:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.371915:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641447424 00000100:00100000:1.0:1788845022.371924:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641447424 00000100:00100000:1.0:1788845022.371973:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 739 00000100:00100000:1.0:1788845022.371981:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641447424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.371997:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.371999:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788845022.372066:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287ac2cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c86 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.372084:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.372088:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845022.372100:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ccc expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.372118:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.372125:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.372149:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.372161:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 21 pid: 11036 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.372170:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660201856:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845022.372225:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845022.374395:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.374480:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641447552 00000100:00100000:0.0:1788845022.374487:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641447552 00010000:00010000:0.0:1788845022.374520:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 22 pid: 11036 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.374556:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 273 00000100:00100000:0.0:1788845022.374563:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:2369:x1875739641447552:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845022.374572:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845022.374593:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 22 pid: 11036 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.374606:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 22 pid: 11036 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.374622:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845022.374627:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.374634:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.374640:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.374665:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.374692:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f777b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660201984:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845022.374763:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00010000:00010000:1.0:1788845022.376795:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac2db lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c94 expref: 22 pid: 11036 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.376820:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660201856:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845022.376851:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.376878:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.376908:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.376912:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5200, rc 0) 00000100:00100000:1.0:1788845022.376965:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641447424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4984us (5601us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.376972:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 739 00000100:00100000:3.0:1788845022.379135:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.379241:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641447680 00000100:00100000:0.0:1788845022.379251:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641447680 00000100:00100000:0.0:1788845022.379330:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 439 00000100:00100000:0.0:1788845022.379342:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641447680:12345-192.168.201.29@tcp:22:fsx.0 00000100:00100000:1.0:1788845022.379800:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f777b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660201984:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845022.379825:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845022.379874:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641447552:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5310us (5487us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.379909:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 273 00000004:00010000:0.0:1788845022.380187:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 49754 -> 49754 00000004:00010000:0.0:1788845022.380192:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00000004:00010000:0.0:1788845022.380194:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:0.0:1788845022.380197:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00000004:00010000:0.0:1788845022.380199:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845022.380236:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641447680:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 896us (1112us total) trans 4294967602 rc 0/0 00000100:00100000:0.0:1788845022.380244:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 439 00000100:00100000:0.0:1788845022.383163:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.383232:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641447808 00000100:00100000:1.0:1788845022.383242:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641447808 00000100:00100000:1.0:1788845022.383293:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 483 00000100:00100000:1.0:1788845022.383303:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641447808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.383321:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.383338:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003a075281/0xb2187301287ac321 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cd3 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.383394:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660202112:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.383478:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.386077:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660202112:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845022.386131:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003a075281/0xb2187301287ac321 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cd3 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.386180:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003a075281/0xb2187301287ac321 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cd3 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.386194:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5000, rc 0) 00000100:00100000:1.0:1788845022.386247:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641447808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2946us (3094us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.386256:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 483 00000100:00100000:0.0:1788845022.389282:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.389425:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641447936 00000100:00100000:1.0:1788845022.389437:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641447936 00000100:00100000:1.0:1788845022.389488:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 484 00000100:00100000:1.0:1788845022.389497:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641447936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.389519:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.389537:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac328 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cda expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.389604:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660202240:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.389681:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.392602:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660202240:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845022.392647:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac328 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cda expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.392673:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac328 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cda expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.392712:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c4200, rc 0) 00000100:00100000:1.0:1788845022.392779:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641447936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3281us (3504us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.392788:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 484 00000100:00100000:3.0:1788845022.400181:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.400308:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641448064 00000100:00100000:2.0:1788845022.400318:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641448064 00000100:00100000:2.0:1788845022.400382:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 740 00000100:00100000:2.0:1788845022.400393:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641448064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.400413:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.400436:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ce1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.400471:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.400501:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce1 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.400537:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.400566:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 12 pid: 5840 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.400581:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f825880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660202368:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845022.400664:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01373aa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.403687:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.403773:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641448192 00000100:00100000:1.0:1788845022.403782:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641448192 00010000:00010000:1.0:1788845022.403826:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 13 pid: 5840 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.403854:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f825880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660202368:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845022.403865:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 274 00000100:00100000:1.0:1788845022.403874:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641448192:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845022.403914:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce1 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.403920:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845022.403933:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.403936:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845022.403939:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce1 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.403949:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845022.404071:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641448064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3680us (3902us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.404080:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 740 00010000:00010000:1.0:1788845022.405134:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 13 pid: 5840 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.405155:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 13 pid: 5840 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.405180:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac31a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ccc expref: 13 pid: 5840 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.405195:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.405201:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce1 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.405231:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce1 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.405242:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce1 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.405470:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce1 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.405487:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660202496:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845022.405583:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.411471:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660202496:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845022.411514:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845022.411581:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641448192:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7706us (7903us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.411592:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 274 00000100:00100000:2.0:1788845022.412073:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.412113:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641448320 00000100:00100000:1.0:1788845022.412120:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641448320 00002000:00100000:1.0:1788845022.412164:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f825880 x1875739641448320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.412174:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f825880 with x1875739641448320 00000100:00100000:1.0:1788845022.412208:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 440 00000100:00100000:1.0:1788845022.412217:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+22:2369:x1875739641448320:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.422132:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+22:2369:x1875739641448320:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 9904us (10048us total) trans 0 rc 53248/53248 00000100:00100000:1.0:1788845022.422147:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 440 00002000:00100000:1.0:1788845022.422163:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f825880 x1875739641448320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 53248/53248 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.422173:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f825880 with x1875739641448320 00000100:00100000:0.0:1788845022.427966:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.428014:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.428122:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641448448 00000100:00100000:3.0:1788845022.428132:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641448448 00000100:00100000:3.0:1788845022.428195:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 741 00000100:00100000:3.0:1788845022.428205:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641448448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.428228:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.428248:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ce8 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.428284:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845022.428297:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 49754 00010000:00010000:3.0:1788845022.428306:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.428321:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.428325:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845022.428329:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845022.428398:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641448448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 193us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.428405:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 741 00000100:00100000:1.0:1788845022.428646:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641448576 00000100:00100000:1.0:1788845022.428651:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641448576 00000100:00100000:1.0:1788845022.428690:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 485 00000100:00100000:1.0:1788845022.428698:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641448576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.428707:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.428725:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac33d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cef expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.428774:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660202624:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.428840:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00002000:00010000:1.0:1788845022.431765:0:7983:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1239042 -> 1986416 00000100:00100000:1.0:1788845022.431799:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660202624:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845022.431837:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac33d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cef expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.431862:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac33d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cef expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.431878:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5200, rc 0) 00000100:00100000:1.0:1788845022.431991:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641448576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3292us (3975us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.432000:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 485 00000100:00100000:3.0:1788845022.435082:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.435178:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641448704 00000100:00100000:0.0:1788845022.435188:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641448704 00000100:00100000:0.0:1788845022.435241:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:0.0:1788845022.435250:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641448704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.435281:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.435297:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ac344 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cf6 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.435354:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660202752:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845022.435435:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.438180:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660202752:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845022.438227:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ac344 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cf6 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.438253:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ac344 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cf6 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.438268:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb000, rc 0) 00000100:00100000:2.0:1788845022.438332:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641448704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3082us (3258us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845022.438340:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:3.0:1788845022.445596:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.445746:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641448832 00000100:00100000:1.0:1788845022.445755:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641448832 00002000:00100000:1.0:1788845022.445802:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1241088->1970175) req@ffff949f0383f800 x1875739641448832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.445812:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383f800 with x1875739641448832 ext(1241088->1970175) 00002000:00010000:1.0:1788845022.445819:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383f800 00000100:00100000:1.0:1788845022.445856:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 280 00000100:00100000:1.0:1788845022.445866:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641448832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.451001:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641448832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 5124us (5407us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.451017:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 280 00002000:00100000:1.0:1788845022.451040:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1241088->1970175) req@ffff949f0383f800 x1875739641448832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.451050:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383f800 with x1875739641448832 ext(1241088->1970175) 00002000:00010000:1.0:1788845022.451057:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383f800 00000100:00100000:0.0:1788845022.455702:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.455797:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641449088 00000100:00100000:1.0:1788845022.455806:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641449088 00000100:00100000:1.0:1788845022.455899:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 742 00000100:00100000:1.0:1788845022.455917:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641449088:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845022.455971:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845022.455997:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac2c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c7f expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.456015:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.456020:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845022.456239:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.456264:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.456273:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.456301:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.456323:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 12 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.456337:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660202880:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845022.456409:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.458608:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660202880:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845022.458640:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.459977:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.460250:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641449216 00000100:00100000:3.0:1788845022.460283:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641449216 00010000:00010000:3.0:1788845022.460320:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 13 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.460355:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 275 00000100:00100000:3.0:1788845022.460365:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641449216:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845022.460377:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845022.460384:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 13 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.460400:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 13 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.460422:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac2e9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c9b expref: 13 pid: 5841 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.460436:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.460442:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.460450:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.460458:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.460482:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845022.460538:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641449216:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 174us (567us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.460565:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 275 00010000:00010000:2.0:1788845022.461061:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.461072:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845022.461188:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967603 00000004:00010000:2.0:1788845022.461239:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 49754 -> 49754 00000004:00010000:2.0:1788845022.461242:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00000004:00010000:2.0:1788845022.461245:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:2.0:1788845022.461252:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00000004:00010000:2.0:1788845022.461267:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:2.0:1788845022.461274:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c9823ef1 reply state 000000006e7a36dc transno 4294967604 ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.461281:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845022.461361:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641449088:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5444us (5669us total) trans 4294967604 rc 0/0 00000100:00100000:2.0:1788845022.461368:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 742 00000020:00080000:2.0:1788845022.461372:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845022.463413:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.463428:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.463434:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.463440:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a0206c5200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845022.463445:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.463452:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.463466:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.463468:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845022.463472:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.463491:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac34b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.466215:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.466464:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641449344 00000100:00100000:3.0:1788845022.466475:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641449344 00000100:00100000:3.0:1788845022.466528:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 743 00000100:00100000:3.0:1788845022.466536:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641449344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.466567:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.466569:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845022.466608:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac32f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce1 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.466625:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.466628:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845022.466640:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7cfd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.466660:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.466669:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.466994:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.467012:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 11 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.467143:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660203008:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845022.467251:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.473535:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.473603:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641449472 00000100:00100000:2.0:1788845022.473612:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641449472 00010000:00010000:2.0:1788845022.473653:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 12 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.473751:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 276 00000100:00100000:2.0:1788845022.473763:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641449472:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845022.473778:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845022.473805:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 12 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.473823:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 12 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.473846:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.473853:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.473903:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.473913:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.473951:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.473965:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660203136:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845022.474066:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788845022.476813:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ac336 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ce8 expref: 12 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.476838:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660203008:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845022.476862:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.476875:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.476877:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845022.476879:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:0.0:1788845022.477183:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641449344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10646us (10974us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.477190:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 743 00000100:00100000:3.0:1788845022.477264:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660203136:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845022.477277:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845022.477299:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641449472:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3539us (3772us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.477307:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 276 00000100:00100000:1.0:1788845022.479235:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.479460:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641449600 00000100:00100000:0.0:1788845022.479467:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641449600 00002000:00010000:0.0:1788845022.479500:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1314316->18446744073709551615) 00002000:00100000:0.0:1788845022.479504:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc6300 with x1875739641449600 ext(1314316->18446744073709551615) 00002000:00010000:0.0:1788845022.479513:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc6300. 00000100:00100000:0.0:1788845022.479577:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 281 00000100:00100000:0.0:1788845022.479584:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641449600:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845022.479850:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1986416 -> 1314316 00002000:00010000:0.0:1788845022.479853:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00002000:00010000:0.0:1788845022.479868:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845022 -> 1788845021 00002000:00010000:0.0:1788845022.479869:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00002000:00010000:0.0:1788845022.479871:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 341 -> 73 00000100:00100000:0.0:1788845022.479951:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641449600:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 366us (720us total) trans 4294967388 rc 0/0 00000100:00100000:0.0:1788845022.479959:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 281 00002000:00010000:0.0:1788845022.479963:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1314316->18446744073709551615) 00002000:00100000:0.0:1788845022.479966:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc6300 with x1875739641449600 ext(1314316->18446744073709551615) 00002000:00010000:0.0:1788845022.479970:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc6300. 00000100:00100000:2.0:1788845022.482519:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.482643:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641449728 00000100:00100000:3.0:1788845022.482652:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641449728 00000100:00100000:3.0:1788845022.482741:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 744 00000100:00100000:3.0:1788845022.482752:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641449728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.482772:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.482793:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac359 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.482943:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac360 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.482958:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac360 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.482969:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac360 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845022.482997:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1314316 00000004:00200000:3.0:1788845022.483002:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845022.483044:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac360 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.483073:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac359 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d04 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.483091:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac360 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d04 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.483099:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.483102:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845022.483105:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:3.0:1788845022.483164:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641449728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 415us (653us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.483185:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 744 00000100:00100000:3.0:1788845022.486298:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.486366:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.486512:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641449856 00000100:00100000:2.0:1788845022.486523:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641449856 00000100:00100000:2.0:1788845022.486591:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 745 00000100:00100000:2.0:1788845022.486601:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641449856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.486623:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.486646:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac367 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d0b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.486686:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.486736:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f825180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660203264:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845022.486820:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.487126:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641449984 00000100:00100000:0.0:1788845022.487132:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641449984 00000100:00100000:0.0:1788845022.487176:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 487 00000100:00100000:0.0:1788845022.487183:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641449984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.487192:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.487203:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ac36e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d12 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.487241:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660203392:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845022.487296:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.491449:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f825180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660203264:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845022.491475:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 49754 00010000:00010000:2.0:1788845022.491493:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac367 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d0b expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.491513:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.491516:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845022.491520:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ac367 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d0b expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.491534:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845022.491655:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641449856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5056us (5288us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845022.491664:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 745 00000100:00100000:0.0:1788845022.491755:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660203392:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845022.491776:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ac36e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d12 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.491793:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ac36e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d12 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.491826:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb400, rc 0) 00000100:00100000:0.0:1788845022.491845:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641449984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4663us (5557us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845022.491852:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 487 00000100:00100000:2.0:1788845022.495521:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.496724:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.497073:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641450240 00000100:00100000:0.0:1788845022.497084:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641450240 00000100:00100000:0.0:1788845022.497149:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 488 00000100:00100000:0.0:1788845022.497159:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641450240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.497176:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.497194:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac375 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d20 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.497252:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660203520:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845022.497335:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.500450:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660203520:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845022.500494:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac375 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d20 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.500521:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ac375 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d20 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.500539:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbbc00, rc 0) 00000100:00100000:0.0:1788845022.500621:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641450240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3463us (3900us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845022.500648:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 488 00000100:00100000:1.0:1788845022.501092:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641450112 00000100:00100000:1.0:1788845022.501100:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641450112 00000100:00100000:1.0:1788845022.501155:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 746 00000100:00100000:1.0:1788845022.501164:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641450112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.501181:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.501199:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac37c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d19 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.501235:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.501280:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660203648:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.501356:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.504111:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660203648:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845022.504139:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 49754 00010000:00010000:1.0:1788845022.504159:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac37c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d19 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.504180:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.504183:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.504188:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac37c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d19 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.504222:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5000, rc 0) 00000100:00100000:1.0:1788845022.504281:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641450112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3117us (8770us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.504289:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 746 00000100:00100000:2.0:1788845022.508242:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.508386:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641450368 00000100:00100000:0.0:1788845022.508397:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641450368 00002000:00100000:0.0:1788845022.508448:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824a80 x1875739641450368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.508458:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824a80 with x1875739641450368 00000100:00100000:0.0:1788845022.508555:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 441 00000100:00100000:0.0:1788845022.508566:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641450368:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.508804:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641450368:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 239us (572us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.508811:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 441 00002000:00100000:0.0:1788845022.508819:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824a80 x1875739641450368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.508826:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824a80 with x1875739641450368 00000100:00100000:3.0:1788845022.514652:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.514849:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641450496 00000100:00100000:0.0:1788845022.514860:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641450496 00002000:00100000:0.0:1788845022.515090:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea8e00 x1875739641450496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.515102:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea8e00 with x1875739641450496 00000100:00100000:0.0:1788845022.515129:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 442 00000100:00100000:0.0:1788845022.515138:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641450496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.515382:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641450496:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 244us (738us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.515390:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 442 00002000:00100000:0.0:1788845022.515411:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea8e00 x1875739641450496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.515419:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea8e00 with x1875739641450496 00000100:00100000:2.0:1788845022.519750:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.519850:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641450624 00000100:00100000:3.0:1788845022.519865:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641450624 00002000:00100000:3.0:1788845022.520111:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f827b80 x1875739641450624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.520122:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f827b80 with x1875739641450624 00000100:00100000:3.0:1788845022.520152:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 443 00000100:00100000:3.0:1788845022.520162:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641450624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.520426:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641450624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 265us (688us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.520435:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 443 00002000:00100000:3.0:1788845022.520441:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f827b80 x1875739641450624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.520449:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f827b80 with x1875739641450624 00000100:00100000:2.0:1788845022.523871:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.524188:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641450752 00000100:00100000:0.0:1788845022.524200:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641450752 00002000:00100000:0.0:1788845022.524254:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824700 x1875739641450752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.524265:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824700 with x1875739641450752 00000100:00100000:0.0:1788845022.524306:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 444 00000100:00100000:0.0:1788845022.524317:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641450752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.524603:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641450752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 287us (741us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.524611:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 444 00002000:00100000:0.0:1788845022.524618:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824700 x1875739641450752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.524626:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824700 with x1875739641450752 00000100:00100000:1.0:1788845022.528153:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.528206:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641450880 00000100:00100000:3.0:1788845022.528216:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641450880 00002000:00100000:3.0:1788845022.528270:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc4a80 x1875739641450880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.528281:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4a80 with x1875739641450880 00000100:00100000:3.0:1788845022.528320:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 445 00000100:00100000:3.0:1788845022.528330:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641450880:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.528661:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641450880:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 330us (515us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.528670:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 445 00002000:00100000:3.0:1788845022.528695:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc4a80 x1875739641450880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.528703:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4a80 with x1875739641450880 00000100:00100000:2.0:1788845022.532151:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.532242:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641451008 00000100:00100000:0.0:1788845022.532250:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641451008 00002000:00100000:0.0:1788845022.532300:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826d80 x1875739641451008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.532311:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826d80 with x1875739641451008 00000100:00100000:0.0:1788845022.532348:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 446 00000100:00100000:0.0:1788845022.532357:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.532947:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 588us (804us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.532972:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 446 00002000:00100000:0.0:1788845022.532981:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826d80 x1875739641451008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.532989:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826d80 with x1875739641451008 00000100:00100000:2.0:1788845022.536805:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.536982:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641451136 00000100:00100000:3.0:1788845022.536992:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641451136 00002000:00100000:3.0:1788845022.537124:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824e00 x1875739641451136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.537135:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824e00 with x1875739641451136 00000100:00100000:3.0:1788845022.537213:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 447 00000100:00100000:3.0:1788845022.537223:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641451136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.537474:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641451136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 251us (679us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.537481:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 447 00002000:00100000:3.0:1788845022.537489:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824e00 x1875739641451136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.537496:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824e00 with x1875739641451136 00000100:00100000:2.0:1788845022.541410:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.541505:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641451264 00000100:00100000:1.0:1788845022.541515:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641451264 00002000:00100000:1.0:1788845022.541574:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824000 x1875739641451264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.541585:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824000 with x1875739641451264 00000100:00100000:1.0:1788845022.541616:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 448 00000100:00100000:1.0:1788845022.541626:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.541874:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 249us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.541912:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 448 00002000:00100000:1.0:1788845022.541920:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824000 x1875739641451264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.541928:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824000 with x1875739641451264 00000100:00100000:2.0:1788845022.551469:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.551552:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641451392 00000100:00100000:0.0:1788845022.551561:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641451392 00002000:00100000:0.0:1788845022.551602:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f825500 x1875739641451392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.551610:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f825500 with x1875739641451392 00000100:00100000:0.0:1788845022.551638:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 449 00000100:00100000:0.0:1788845022.551644:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.551935:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 288us (471us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.551942:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 449 00002000:00100000:0.0:1788845022.551949:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f825500 x1875739641451392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.551957:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f825500 with x1875739641451392 00000100:00100000:2.0:1788845022.554238:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.554308:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641451520 00000100:00100000:1.0:1788845022.554316:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641451520 00002000:00100000:1.0:1788845022.554351:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826680 x1875739641451520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.554358:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826680 with x1875739641451520 00000100:00100000:1.0:1788845022.554384:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 450 00000100:00100000:1.0:1788845022.554404:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.554593:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 190us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.554598:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 450 00002000:00100000:1.0:1788845022.554602:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826680 x1875739641451520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.554607:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826680 with x1875739641451520 00000100:00100000:2.0:1788845022.556663:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.556758:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641451648 00000100:00100000:0.0:1788845022.556767:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641451648 00002000:00100000:0.0:1788845022.556809:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826a00 x1875739641451648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.556817:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826a00 with x1875739641451648 00000100:00100000:0.0:1788845022.556845:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 451 00000100:00100000:0.0:1788845022.556854:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.557148:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 294us (491us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.557153:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 451 00002000:00100000:0.0:1788845022.557159:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826a00 x1875739641451648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.557164:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826a00 with x1875739641451648 00000100:00100000:2.0:1788845022.559353:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.559475:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641451776 00000100:00100000:1.0:1788845022.559484:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641451776 00002000:00100000:1.0:1788845022.559532:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684380 x1875739641451776/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.559613:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684380 with x1875739641451776 00000100:00100000:1.0:1788845022.559654:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 452 00000100:00100000:1.0:1788845022.559664:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641451776:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845022.564771:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 49754 -> 1007616 00000004:00010000:3.0:1788845022.564779:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00000004:00010000:3.0:1788845022.564781:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845021 -> 1788845021 00000004:00010000:3.0:1788845022.564783:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00000004:00010000:3.0:1788845022.564784:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845022.564849:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641451776:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5183us (5501us total) trans 4294967605 rc 0/0 00000100:00100000:3.0:1788845022.564857:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 452 00002000:00100000:3.0:1788845022.564867:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684380 x1875739641451776/t4294967605(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/448 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.564919:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684380 with x1875739641451776 00000100:00100000:2.0:1788845022.567250:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.567317:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641451904 00000100:00100000:1.0:1788845022.567327:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641451904 00000100:00100000:1.0:1788845022.567435:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 747 00000100:00100000:1.0:1788845022.567447:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641451904:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845022.591918:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967602 is committed 00000001:00080000:3.0:1788845022.591934:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967598 is committed 00000001:00080000:3.0:1788845022.591942:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967603 is committed 00000001:00080000:3.0:1788845022.591950:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967604 is committed 00000001:00080000:3.0:1788845022.591965:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967605 is committed 00000100:00100000:0.0:1788845022.592124:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641451904:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 24673us (24874us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.592137:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 747 00000100:00100000:2.0:1788845022.595849:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.596434:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641452032 00000100:00100000:3.0:1788845022.596445:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641452032 00000100:00100000:3.0:1788845022.596571:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 748 00000100:00100000:3.0:1788845022.596583:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641452032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.596604:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.596625:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac383 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.596713:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.596728:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.596739:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845022.596768:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1314316 00000004:00200000:3.0:1788845022.596773:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845022.596791:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.596815:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac383 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.596845:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.596853:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.596856:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845022.596858:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:3.0:1788845022.596969:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641452032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 388us (1130us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.596976:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 748 00000100:00100000:2.0:1788845022.600053:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.600115:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.600266:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641452160 00000100:00100000:1.0:1788845022.600276:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641452160 00000100:00100000:1.0:1788845022.600341:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 749 00000100:00100000:1.0:1788845022.600351:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641452160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.600375:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.600396:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac391 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d2e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.600445:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.600486:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660203776:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.600578:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845022.600642:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641452288 00000100:00100000:3.0:1788845022.600647:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641452288 00000100:00100000:3.0:1788845022.600687:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 489 00000100:00100000:3.0:1788845022.600695:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641452288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.600704:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.600715:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac398 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d35 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.600751:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beaad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660203904:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845022.600780:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00002000:00010000:3.0:1788845022.605101:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845021 -> 1788845022 00000004:00010000:1.0:1788845022.605222:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845021 -> 1788845022 00000100:00100000:1.0:1788845022.605247:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660203776:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845022.605273:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845022.605292:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac391 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d2e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.605315:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.605318:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.605323:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac391 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d2e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.605339:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5000, rc 0) 00000100:00100000:1.0:1788845022.605460:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641452160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5110us (5343us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.605469:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 749 00000100:00100000:0.0:1788845022.605475:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beaad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660203904:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845022.605498:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac398 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d35 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.605515:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ac398 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d35 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.605524:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:0.0:1788845022.605545:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641452288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4852us (5504us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845022.605552:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 489 00000100:00100000:2.0:1788845022.609802:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.609932:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641452416 00000100:00100000:1.0:1788845022.609943:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641452416 00000100:00100000:1.0:1788845022.610021:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 750 00000100:00100000:1.0:1788845022.610043:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641452416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.610067:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.610102:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d3c expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.610137:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.610148:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.610183:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.610202:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 14 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.610217:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660204032:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845022.610304:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.614697:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.614778:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641452544 00000100:00100000:3.0:1788845022.614788:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641452544 00010000:00010000:3.0:1788845022.614834:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 15 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.614878:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 277 00000100:00100000:1.0:1788845022.614981:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660204032:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845022.615033:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.615051:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.615055:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.615060:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.615071:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5200, rc 0) 00000100:00100000:1.0:1788845022.615139:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641452416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5099us (5346us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845022.615147:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 750 00000100:00100000:3.0:1788845022.615278:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641452544:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845022.615293:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845022.615317:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 15 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.615335:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 15 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.615359:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac352 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7cfd expref: 15 pid: 5839 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.615373:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.615377:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.615388:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.615397:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.615431:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.615446:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660204160:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845022.615498:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.618656:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660204160:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845022.618686:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845022.618746:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641452544:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3469us (4059us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.626151:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.626289:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641452672 00000100:00100000:2.0:1788845022.626300:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641452672 00002000:00100000:2.0:1788845022.626351:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beaaa00 x1875739641452672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.626364:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beaaa00 with x1875739641452672 00000100:00100000:2.0:1788845022.626404:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 453 00000100:00100000:2.0:1788845022.626414:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641452672:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.626640:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 277 00000100:00100000:2.0:1788845022.644796:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641452672:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 18374us (18646us total) trans 0 rc 954368/954368 00000100:00100000:2.0:1788845022.644811:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 453 00002000:00100000:2.0:1788845022.644823:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beaaa00 x1875739641452672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 954368/954368 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.644833:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beaaa00 with x1875739641452672 00000100:00100000:2.0:1788845022.648973:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.649064:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641452800 00000100:00100000:0.0:1788845022.649074:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641452800 00002000:00100000:0.0:1788845022.649123:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684700 x1875739641452800/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.649152:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684700 with x1875739641452800 00000100:00100000:0.0:1788845022.649241:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 454 00000100:00100000:0.0:1788845022.649252:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641452800:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845022.653299:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1007616 -> 1007616 00000004:00010000:0.0:1788845022.653309:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00000004:00010000:0.0:1788845022.653312:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845022 -> 1788845021 00000004:00010000:0.0:1788845022.653314:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00000004:00010000:0.0:1788845022.653316:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 198 00000100:00100000:0.0:1788845022.654236:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641452800:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4978us (5267us total) trans 4294967606 rc 0/0 00000100:00100000:0.0:1788845022.654249:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 454 00002000:00100000:0.0:1788845022.654264:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684700 x1875739641452800/t4294967606(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:128/0 lens 488/448 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.654274:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684700 with x1875739641452800 00000100:00100000:2.0:1788845022.657555:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.657750:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641452928 00000100:00100000:0.0:1788845022.657760:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641452928 00000100:00100000:0.0:1788845022.657824:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 751 00000100:00100000:0.0:1788845022.657834:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641452928:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:0.0:1788845022.732035:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641452928:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 74184us (74482us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.732051:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 751 00000001:00080000:3.0:1788845022.732492:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967606 is committed 00000100:00100000:3.0:1788845022.735820:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.735974:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641453056 00000100:00100000:2.0:1788845022.735984:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641453056 00000100:00100000:2.0:1788845022.736061:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 752 00000100:00100000:2.0:1788845022.736071:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641453056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.736092:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.736114:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ac3a6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d43 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.736173:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.736214:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03457480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660204288:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845022.736311:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000004:00010000:2.0:1788845022.748047:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845021 -> 1788845022 00000004:00010000:2.0:1788845022.748061:0:5841:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 198 -> 326 00000100:00100000:2.0:1788845022.748087:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03457480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660204288:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845022.748114:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:2.0:1788845022.748134:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ac3a6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d43 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.748155:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.748158:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845022.748163:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ac3a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d43 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.748182:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845022.748248:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641453056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12178us (12435us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845022.748256:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 752 00000100:00100000:3.0:1788845022.756175:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.756238:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641453184 00000100:00100000:1.0:1788845022.756248:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641453184 00000100:00100000:1.0:1788845022.756314:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 753 00000100:00100000:1.0:1788845022.756325:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641453184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.756347:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.756368:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac3ad lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d4a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.756413:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.756453:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660204416:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.756533:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.759042:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660204416:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845022.759072:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845022.759092:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac3ad lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d4a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.759136:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.759139:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.759145:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac3ad lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d4a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.759162:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c4200, rc 0) 00000100:00100000:1.0:1788845022.759225:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641453184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2902us (3060us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.759233:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 753 00000100:00100000:2.0:1788845022.762971:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.763872:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641453312 00000100:00100000:1.0:1788845022.763899:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641453312 00000100:00100000:1.0:1788845022.763971:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 754 00000100:00100000:1.0:1788845022.763984:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641453312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.764013:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.764048:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac3b4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d51 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.764094:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.764132:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660204544:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.764213:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.770823:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660204544:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845022.770857:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:0.0:1788845022.771096:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac3b4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d51 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.771120:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845022.771123:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845022.771128:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac3b4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d51 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.771144:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5000, rc 0) 00000100:00100000:0.0:1788845022.771209:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641453312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7225us (8248us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845022.771217:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 754 00000100:00100000:3.0:1788845022.775249:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.775351:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641453440 00000100:00100000:0.0:1788845022.775362:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641453440 00000100:00100000:0.0:1788845022.775429:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 755 00000100:00100000:0.0:1788845022.775440:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641453440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.775464:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.775488:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac3bb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d58 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.775534:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.775591:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660204672:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845022.775675:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.779101:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660204672:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845022.779133:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:0.0:1788845022.779167:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac3bb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d58 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.779188:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845022.779191:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845022.779196:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac3bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d58 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.779214:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:0.0:1788845022.779286:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641453440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3847us (4047us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845022.779295:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 755 00000100:00100000:2.0:1788845022.784159:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.785119:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641453568 00000100:00100000:3.0:1788845022.785135:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641453568 00000100:00100000:3.0:1788845022.785207:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 756 00000100:00100000:3.0:1788845022.785219:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641453568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845022.785243:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845022.785282:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac3c2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d5f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.785339:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845022.785388:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660204800:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845022.785481:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.788624:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660204800:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845022.788653:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:2.0:1788845022.788794:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac3c2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d5f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.788817:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.788820:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845022.788825:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ac3c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d5f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.788839:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:2.0:1788845022.788940:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641453568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3723us (4787us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845022.788949:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 756 00000100:00100000:2.0:1788845022.795993:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.796070:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641453696 00000100:00100000:1.0:1788845022.796081:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641453696 00000100:00100000:1.0:1788845022.796146:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 757 00000100:00100000:1.0:1788845022.796157:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641453696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.796188:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.796208:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac3c9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d66 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.796243:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.796375:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660204928:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.796454:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.799002:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660204928:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845022.799039:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845022.799060:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac3c9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d66 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.799079:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.799082:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.799087:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287ac3c9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d66 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.799104:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c4200, rc 0) 00000100:00100000:1.0:1788845022.799168:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641453696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3013us (3187us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.799176:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 757 00010000:00010000:1.0:1788845022.803214:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845022.804760:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.809794:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641453824 00000100:00100000:0.0:1788845022.809810:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641453824 00000100:00100000:1.0:1788845022.810330:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 758 00000100:00100000:1.0:1788845022.810345:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641453824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.810385:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.810408:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac3d0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d6d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.810448:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.810506:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660205056:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.810624:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.814288:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660205056:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845022.814317:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845022.814334:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac3d0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d6d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.814353:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.814356:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.814362:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287ac3d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d6d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.814377:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5000, rc 0) 00000100:00100000:1.0:1788845022.814500:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641453824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4145us (9734us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.814509:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 758 00000100:00100000:2.0:1788845022.820184:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.820338:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641453952 00000100:00100000:0.0:1788845022.820348:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641453952 00000100:00100000:0.0:1788845022.820466:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:0.0:1788845022.820478:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641453952:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845022.820815:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1007616 -> 1007616 00000004:00010000:0.0:1788845022.820820:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845022 -> 1788845022 00000004:00010000:0.0:1788845022.820823:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845022 -> 1788845022 00000004:00010000:0.0:1788845022.820825:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845022 -> 1788845022 00000004:00010000:0.0:1788845022.820827:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845022.821196:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641453952:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 718us (1020us total) trans 4294967607 rc 0/0 00000100:00100000:0.0:1788845022.821205:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:2.0:1788845022.825790:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.825873:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641454080 00000100:00100000:0.0:1788845022.825934:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641454080 00000100:00100000:0.0:1788845022.825996:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 490 00000100:00100000:0.0:1788845022.826154:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641454080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.826176:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.826195:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ac3d7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d74 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.826257:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660205184:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845022.826349:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.829934:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660205184:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845022.829981:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ac3d7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d74 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.830144:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ac3d7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d74 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.830162:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:2.0:1788845022.830234:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641454080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4086us (4455us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845022.830242:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 490 00000100:00100000:2.0:1788845022.834172:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.836423:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641454208 00000100:00100000:0.0:1788845022.836435:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641454208 00000100:00100000:0.0:1788845022.836696:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 491 00000100:00100000:0.0:1788845022.836709:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641454208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.836728:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.836745:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cb10f58/0xb2187301287ac3de lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d7b expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.836803:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660205312:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845022.836914:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.839274:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660205312:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845022.839316:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cb10f58/0xb2187301287ac3de lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d7b expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.839340:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cb10f58/0xb2187301287ac3de lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d7b expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.839375:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbae00, rc 0) 00000100:00100000:0.0:1788845022.839427:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641454208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2721us (5264us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845022.839435:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 491 00000100:00100000:2.0:1788845022.844271:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.844760:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641454336 00000100:00100000:0.0:1788845022.844771:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641454336 00000100:00100000:0.0:1788845022.844826:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 759 00000100:00100000:0.0:1788845022.844835:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641454336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845022.844860:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845022.844880:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac3e5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d82 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.844958:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.845170:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660205440:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845022.845321:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.848642:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660205440:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845022.848670:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845022.848689:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac3e5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d82 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.848732:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845022.848735:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845022.848741:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac3e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d82 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.848758:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb600, rc 0) 00000100:00100000:1.0:1788845022.848823:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641454336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3988us (4568us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.848832:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 759 00000100:00100000:3.0:1788845022.856842:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.857303:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641454464 00000100:00100000:2.0:1788845022.857314:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641454464 00000100:00100000:2.0:1788845022.857373:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 760 00000100:00100000:2.0:1788845022.857382:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641454464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.857402:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.857422:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d89 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.857449:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.857459:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.857489:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.857504:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 11 pid: 11036 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845022.857517:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03455880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660205568:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845022.857600:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845022.862263:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03455880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660205568:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845022.862304:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.862325:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845022.862329:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845022.862334:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845022.862366:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e800, rc 0) 00000100:00100000:3.0:1788845022.862434:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641454464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5052us (5601us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845022.862443:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 760 00000100:00100000:3.0:1788845022.863237:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.863331:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641454592 00000100:00100000:0.0:1788845022.863340:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641454592 00010000:00010000:0.0:1788845022.863378:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 12 pid: 11036 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.863415:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 278 00000100:00100000:0.0:1788845022.863424:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008915c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641454592:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845022.863437:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845022.863470:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 12 pid: 11036 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.863488:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 12 pid: 11036 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.863513:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287ac39f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d3c expref: 12 pid: 11036 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.863530:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845022.863536:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.863560:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.863568:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845022.863607:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845022.863619:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660205696:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845022.863691:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845022.868096:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660205696:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845022.868130:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845022.868187:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008915c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641454592:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4763us (4952us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845022.868213:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 278 00000100:00100000:2.0:1788845022.870749:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.871140:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641454720 00000100:00100000:3.0:1788845022.871152:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641454720 00002000:00100000:3.0:1788845022.871203:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454a80 x1875739641454720/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.871215:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454a80 with x1875739641454720 00000100:00100000:3.0:1788845022.871263:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 456 00000100:00100000:3.0:1788845022.871274:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641454720:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.871494:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641454720:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 221us (753us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845022.871502:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 456 00002000:00100000:3.0:1788845022.871510:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454a80 x1875739641454720/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.871518:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454a80 with x1875739641454720 00000100:00100000:3.0:1788845022.875309:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.875746:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641454848 00000100:00100000:2.0:1788845022.875765:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641454848 00002000:00100000:2.0:1788845022.875819:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beaad80 x1875739641454848/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.875831:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beaad80 with x1875739641454848 00000100:00100000:2.0:1788845022.875875:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 457 00000100:00100000:2.0:1788845022.875938:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641454848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845022.876420:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641454848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 535us (1122us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845022.876428:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 457 00002000:00100000:2.0:1788845022.876436:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beaad80 x1875739641454848/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.876443:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beaad80 with x1875739641454848 00000100:00100000:2.0:1788845022.881737:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.881836:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641454976 00000100:00100000:3.0:1788845022.881847:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641454976 00002000:00100000:3.0:1788845022.882114:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03455c00 x1875739641454976/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.882418:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455c00 with x1875739641454976 00000100:00100000:3.0:1788845022.882484:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 458 00000100:00100000:3.0:1788845022.882500:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641454976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.882812:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641454976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 317us (1084us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845022.882820:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 458 00002000:00100000:3.0:1788845022.882827:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03455c00 x1875739641454976/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.882835:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455c00 with x1875739641454976 00000100:00100000:2.0:1788845022.888752:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.889081:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641455104 00000100:00100000:3.0:1788845022.889092:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641455104 00002000:00100000:3.0:1788845022.889139:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03455880 x1875739641455104/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.889150:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455880 with x1875739641455104 00000100:00100000:3.0:1788845022.889190:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 459 00000100:00100000:3.0:1788845022.889199:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641455104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.889506:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641455104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 306us (762us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845022.889515:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 459 00002000:00100000:3.0:1788845022.889523:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03455880 x1875739641455104/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.889531:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455880 with x1875739641455104 00000100:00100000:2.0:1788845022.894871:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.894995:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641455232 00000100:00100000:3.0:1788845022.895013:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641455232 00002000:00100000:3.0:1788845022.895068:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918e00 x1875739641455232/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.895098:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918e00 with x1875739641455232 00000100:00100000:3.0:1788845022.895171:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 460 00000100:00100000:3.0:1788845022.895181:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641455232:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.895403:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641455232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 223us (541us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845022.895410:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 460 00002000:00100000:3.0:1788845022.895417:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918e00 x1875739641455232/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.895424:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918e00 with x1875739641455232 00000100:00100000:2.0:1788845022.898869:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.899059:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641455360 00000100:00100000:0.0:1788845022.899069:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641455360 00002000:00100000:0.0:1788845022.899117:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a00891bb80 x1875739641455360/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.899128:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a00891bb80 with x1875739641455360 00000100:00100000:0.0:1788845022.899166:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 461 00000100:00100000:0.0:1788845022.899175:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641455360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.899431:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641455360:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (571us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845022.899439:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 461 00002000:00100000:0.0:1788845022.899446:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a00891bb80 x1875739641455360/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.899454:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a00891bb80 with x1875739641455360 00000100:00100000:2.0:1788845022.902821:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.902956:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641455488 00000100:00100000:0.0:1788845022.902965:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641455488 00002000:00100000:0.0:1788845022.903084:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918a80 x1875739641455488/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.903095:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918a80 with x1875739641455488 00000100:00100000:0.0:1788845022.903131:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 462 00000100:00100000:0.0:1788845022.903140:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641455488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.903473:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641455488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 333us (658us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845022.903482:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 462 00002000:00100000:0.0:1788845022.903505:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918a80 x1875739641455488/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.903513:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918a80 with x1875739641455488 00000100:00100000:2.0:1788845022.908308:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845022.908411:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641455616 00000100:00100000:0.0:1788845022.908421:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641455616 00002000:00100000:0.0:1788845022.908474:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919f80 x1875739641455616/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.908485:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919f80 with x1875739641455616 00000100:00100000:0.0:1788845022.908681:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 463 00000100:00100000:0.0:1788845022.908692:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641455616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845022.908980:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641455616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 288us (680us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845022.908988:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 463 00002000:00100000:0.0:1788845022.908995:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919f80 x1875739641455616/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845022.909220:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919f80 with x1875739641455616 00000100:00100000:2.0:1788845022.912570:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.912726:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641455744 00000100:00100000:3.0:1788845022.912738:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641455744 00002000:00100000:3.0:1788845022.912786:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919500 x1875739641455744/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.912796:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919500 with x1875739641455744 00000100:00100000:3.0:1788845022.912836:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 464 00000100:00100000:3.0:1788845022.912845:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641455744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.913313:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641455744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 467us (752us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845022.913320:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 464 00002000:00100000:3.0:1788845022.913326:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919500 x1875739641455744/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.913333:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919500 with x1875739641455744 00000100:00100000:1.0:1788845022.917586:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.917926:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641455872 00000100:00100000:2.0:1788845022.917937:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641455872 00002000:00100000:2.0:1788845022.917987:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5c00 x1875739641455872/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.918131:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5c00 with x1875739641455872 00000100:00100000:2.0:1788845022.918172:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 465 00000100:00100000:2.0:1788845022.918183:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641455872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845022.918519:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641455872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 336us (942us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845022.918527:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 465 00002000:00100000:2.0:1788845022.918534:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5c00 x1875739641455872/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.918553:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5c00 with x1875739641455872 00000100:00100000:2.0:1788845022.923115:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.923255:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641456000 00000100:00100000:3.0:1788845022.923280:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641456000 00002000:00100000:3.0:1788845022.923329:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918700 x1875739641456000/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.923340:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918700 with x1875739641456000 00000100:00100000:3.0:1788845022.923379:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 466 00000100:00100000:3.0:1788845022.923388:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641456000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845022.923677:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641456000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 289us (572us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845022.923685:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 466 00002000:00100000:3.0:1788845022.923692:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918700 x1875739641456000/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.923699:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918700 with x1875739641456000 00000100:00100000:3.0:1788845022.927439:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.927931:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641456128 00000100:00100000:2.0:1788845022.927941:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641456128 00002000:00100000:2.0:1788845022.927996:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445880 x1875739641456128/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.928027:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445880 with x1875739641456128 00000100:00100000:2.0:1788845022.928069:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 467 00000100:00100000:2.0:1788845022.928079:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641456128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845022.928493:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641456128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 414us (1065us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845022.928502:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 467 00002000:00100000:2.0:1788845022.928509:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445880 x1875739641456128/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 488/4536 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845022.928518:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445880 with x1875739641456128 00000100:00100000:1.0:1788845022.933455:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.933571:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845022.933764:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641456256 00000100:00100000:2.0:1788845022.933775:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641456256 00000100:00100000:2.0:1788845022.933834:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 761 00000100:00100000:2.0:1788845022.933845:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641456256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845022.933929:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845022.933950:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac3f3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d90 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.933985:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac3f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d90 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845022.934153:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:2.0:1788845022.934164:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac3f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d90 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.934179:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845022.934181:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845022.934186:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845022.934245:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641456256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 402us (673us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845022.934253:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 761 00000100:00100000:0.0:1788845022.935993:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641456384 00000100:00100000:0.0:1788845022.936085:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641456384 00000100:00100000:1.0:1788845022.936285:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 492 00000100:00100000:1.0:1788845022.936296:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641456384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.936313:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.936330:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac3fa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d97 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.936383:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660205824:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.936473:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0104630c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845022.939835:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660205824:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845022.939968:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac3fa lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d97 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.939994:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000080112/0xb2187301287ac3fa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d97 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845022.940095:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c4200, rc 0) 00000100:00100000:2.0:1788845022.940203:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641456384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3907us (6756us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845022.940227:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 492 00000100:00100000:3.0:1788845022.943951:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845022.944190:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641456512 00000100:00100000:1.0:1788845022.944201:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641456512 00000100:00100000:1.0:1788845022.944261:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 493 00000100:00100000:1.0:1788845022.944272:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641456512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845022.944308:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845022.944326:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac401 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7d9e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845022.944381:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d92d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660205952:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845022.944460:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010463a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845022.947490:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d92d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660205952:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845022.947535:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac401 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d9e expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.947575:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ca2b623e/0xb2187301287ac401 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d9e expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845022.947590:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5200, rc 0) 00000100:00100000:1.0:1788845022.947655:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641456512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3384us (3712us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845022.947663:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 493 00000100:00100000:2.0:1788845022.957203:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845022.957295:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641456640 00000100:00100000:3.0:1788845022.957305:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641456640 00002000:00100000:3.0:1788845022.957353:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919c00 x1875739641456640/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 504/0 e 0 to 0 dl 1788845033 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845022.957363:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919c00 with x1875739641456640 00000100:00100000:3.0:1788845022.957404:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 468 00000100:00100000:3.0:1788845022.957413:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641456640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845022.994369:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641456640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 36944us (37165us total) trans 0 rc 958464/958464 00000100:00100000:1.0:1788845022.994384:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 468 00002000:00100000:1.0:1788845022.994399:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919c00 x1875739641456640/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:128/0 lens 504/440 e 0 to 0 dl 1788845033 ref 1 fl Complete:/600/0 rc 958464/958464 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845022.994410:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919c00 with x1875739641456640 00000100:00100000:2.0:1788845023.000221:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.000361:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641456896 00000100:00100000:3.0:1788845023.000371:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641456896 00000100:00100000:3.0:1788845023.000490:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 762 00000100:00100000:3.0:1788845023.000502:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641456896:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845023.000596:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845023.000626:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac360 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d04 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.000646:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845023.000652:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845023.001211:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.001228:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.001237:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.001404:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.001426:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 14 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.001441:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660206080:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845023.001574:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845023.009370:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.009496:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660206080:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845023.009522:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.009698:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641457024 00000100:00100000:0.0:1788845023.009706:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641457024 00010000:00010000:0.0:1788845023.009788:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 15 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.009829:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 279 00000100:00100000:0.0:1788845023.009839:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641457024:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845023.009855:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845023.009875:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 15 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.009922:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 15 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.009944:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ac38a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d27 expref: 15 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.009959:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.009963:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.009973:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.009980:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.010144:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845023.010192:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641457024:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 354us (830us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.010199:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 279 00010000:00010000:2.0:1788845023.010273:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.010280:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845023.010394:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967608 00000004:00010000:2.0:1788845023.010442:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1007616 -> 1007616 00000004:00010000:2.0:1788845023.010446:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845022 -> 1788845023 00000004:00010000:2.0:1788845023.010448:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845022 -> 1788845022 00000004:00010000:2.0:1788845023.010450:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845022 -> 1788845023 00000004:00010000:2.0:1788845023.010452:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:2.0:1788845023.010458:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005ad2ba05 reply state 00000000635838a4 transno 4294967609 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.010466:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845023.010527:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641456896:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 10029us (10316us total) trans 4294967609 rc 0/0 00000100:00100000:2.0:1788845023.010726:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 762 00000020:00080000:2.0:1788845023.010731:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00080000:3.0:1788845023.014099:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845023.014633:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.014777:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641457152 00000100:00100000:2.0:1788845023.014787:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641457152 00000100:00100000:2.0:1788845023.014853:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 763 00000100:00100000:2.0:1788845023.014862:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641457152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.014910:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.014913:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845023.014953:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac3f3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d90 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.014973:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845023.014977:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845023.014989:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7da5 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.015031:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.015042:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.015071:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.015088:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 13 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.015100:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660206208:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845023.015164:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562cf00 going to sleep for 11 seconds 00000100:00080000:3.0:1788845023.015567:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845023.021404:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.021922:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641457280 00000100:00100000:3.0:1788845023.021933:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641457280 00010000:00010000:3.0:1788845023.021977:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 14 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.022157:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 280 00000100:00100000:3.0:1788845023.022169:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641457280:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845023.022184:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845023.022208:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 14 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.022225:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 14 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.022247:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845023.022253:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.022275:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.022283:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.022321:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.022355:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660206336:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845023.022446:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00010000:00010000:1.0:1788845023.023395:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac3ec lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7d89 expref: 14 pid: 5841 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.023414:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660206208:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845023.023443:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.023457:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.023459:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.023462:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:1.0:1788845023.023503:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641457152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8641us (8876us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.023512:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 763 00010000:00010000:0.0:1788845023.028574:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.028591:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.028598:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.028621:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d15000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845023.028627:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.028634:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.028652:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.028654:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845023.028658:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.028665:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ac408 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.029948:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.030041:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641457408 00000100:00100000:2.0:1788845023.030050:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641457408 00000100:00100000:2.0:1788845023.030131:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 494 00000100:00100000:2.0:1788845023.030143:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641457408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.030161:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.030180:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7dac expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.030209:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.030218:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(12 10) 00010000:00010000:2.0:1788845023.030222:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:2.0:1788845023.030226:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1788845023.030445:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1788845023.030484:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.030503:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 6 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.030519:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a48e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660206464:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845023.030621:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845023.036755:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660206336:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845023.036784:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845023.037442:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.037805:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a48e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660206464:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845023.037833:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.037871:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:2.0:1788845023.037955:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641457408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7814us (8014us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845023.037964:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 494 00000100:00100000:3.0:1788845023.039074:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641457536 00000100:00100000:3.0:1788845023.039088:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641457536 00010000:00010000:3.0:1788845023.039148:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 7 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.039185:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 281 00000100:00100000:3.0:1788845023.039194:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641457536:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845023.039208:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845023.039276:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 7 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.039295:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 7 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.039321:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a26691c9/0xb2187301287ac2a3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7c71 expref: 7 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.039337:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:3.0:1788845023.039342:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:3.0:1788845023.039350:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.039373:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.039412:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.039428:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660206592:192.168.201.29@tcp:105: 00000100:00080000:1.0:1788845023.039522:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845023.041321:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.041388:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641457280:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 19214us (19988us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.041400:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 280 00000100:00100000:1.0:1788845023.046774:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.046857:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641457664 00000100:00100000:2.0:1788845023.046866:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641457664 00002000:00010000:2.0:1788845023.046936:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1604682->18446744073709551615) 00002000:00100000:2.0:1788845023.046940:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d90380 with x1875739641457664 ext(1604682->18446744073709551615) 00002000:00010000:2.0:1788845023.046951:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d90380. 00000100:00100000:2.0:1788845023.046982:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 282 00000100:00100000:2.0:1788845023.046992:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641457664:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845023.047237:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1314316 -> 1604682 00002000:00010000:2.0:1788845023.047241:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845022 -> 1788845023 00002000:00010000:2.0:1788845023.047243:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845022 -> 1788845021 00002000:00010000:2.0:1788845023.047246:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845022 -> 1788845023 00002000:00010000:2.0:1788845023.047248:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 73 -> 73 00000100:00100000:2.0:1788845023.047318:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641457664:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 326us (551us total) trans 4294967389 rc 0/0 00000100:00100000:2.0:1788845023.047326:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 282 00002000:00010000:2.0:1788845023.047330:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1604682->18446744073709551615) 00002000:00100000:2.0:1788845023.047332:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d90380 with x1875739641457664 ext(1604682->18446744073709551615) 00002000:00010000:2.0:1788845023.047336:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d90380. 00000100:00100000:0.0:1788845023.050231:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660206592:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845023.050259:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845023.051637:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641457536:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 12432us (14188us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.051652:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 281 00000100:00100000:3.0:1788845023.056831:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.056931:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.057040:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641457920 00000100:00100000:0.0:1788845023.057051:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641457920 00002000:00100000:0.0:1788845023.057106:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445500 x1875739641457920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845023.057117:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445500 with x1875739641457920 00000100:00100000:0.0:1788845023.057159:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 469 00000100:00100000:0.0:1788845023.057186:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641457920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.057756:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641457792 00000100:00100000:2.0:1788845023.057769:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641457792 00002000:00100000:2.0:1788845023.057799:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1605631) req@ffff949f03444a80 x1875739641457792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.057810:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03444a80 with x1875739641457792 ext(1048576->1605631) 00002000:00010000:2.0:1788845023.057814:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03444a80 00000100:00100000:2.0:1788845023.057845:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 283 00000100:00100000:2.0:1788845023.057854:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641457792:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845023.065519:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641457920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8321us (8687us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.065534:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 469 00002000:00100000:1.0:1788845023.065675:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445500 x1875739641457920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/440 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845023.065687:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445500 with x1875739641457920 00000100:00100000:2.0:1788845023.075876:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641457792:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 18012us (18933us total) trans 0 rc 557056/557056 00000100:00100000:2.0:1788845023.075923:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 283 00002000:00100000:2.0:1788845023.075937:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1605631) req@ffff949f03444a80 x1875739641457792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/440 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 557056/557056 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.075947:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03444a80 with x1875739641457792 ext(1048576->1605631) 00002000:00010000:2.0:1788845023.075954:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03444a80 00000100:00100000:1.0:1788845023.080973:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.081513:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641458048 00000100:00100000:2.0:1788845023.081524:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641458048 00002000:00100000:2.0:1788845023.081588:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1604681) req@ffff949f02360a80 x1875739641458048/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 11832/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.081599:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360a80 with x1875739641458048 ext(1593344->1604681) 00002000:00010000:2.0:1788845023.081606:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360a80 00000100:00100000:2.0:1788845023.081636:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 284 00000100:00100000:2.0:1788845023.081647:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641458048:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845023.082420:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641458048:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 771us (1455us total) trans 4294967390 rc 0/0 00000100:00100000:2.0:1788845023.082439:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 284 00002000:00100000:2.0:1788845023.082447:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1604681) req@ffff949f02360a80 x1875739641458048/t4294967390(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 11832/448 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.082455:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360a80 with x1875739641458048 ext(1593344->1604681) 00002000:00010000:2.0:1788845023.082459:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360a80 00000100:00100000:1.0:1788845023.085474:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.085631:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641458176 00000100:00100000:0.0:1788845023.085641:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641458176 00000100:00100000:0.0:1788845023.085738:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 495 00000100:00100000:0.0:1788845023.085749:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641458176:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845023.114228:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967388 is committed 00000001:00080000:1.0:1788845023.114249:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967390 is committed 00000100:00100000:0.0:1788845023.114397:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641458176:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 28645us (28927us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.114409:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 495 00000100:00100000:0.0:1788845023.117670:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.117768:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641458304 00000100:00100000:1.0:1788845023.117779:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641458304 00000100:00100000:1.0:1788845023.117871:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 764 00000100:00100000:1.0:1788845023.117905:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641458304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.117925:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845023.117945:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ac41d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7db3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.118146:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac424 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.118161:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac424 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.118172:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac424 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845023.118197:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1604682 00000004:00200000:1.0:1788845023.118201:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845023.118218:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac424 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.118246:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ac41d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7db3 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.118265:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac424 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7db3 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.118293:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.118296:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845023.118300:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1be00, rc 0) 00000100:00100000:1.0:1788845023.118382:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641458304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 502us (724us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.118389:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 764 00000100:00100000:3.0:1788845023.122101:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.122919:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641458560 00000100:00100000:2.0:1788845023.122932:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641458560 00000100:00100000:2.0:1788845023.123195:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 765 00000100:00100000:2.0:1788845023.123208:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641458560:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845023.123286:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845023.123320:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac424 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7db3 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.123342:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845023.123348:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845023.123740:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.123758:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.123768:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845023.123827:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967610 00000004:00010000:2.0:1788845023.123854:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1007616 -> 1007616 00000004:00010000:2.0:1788845023.123857:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845023.123875:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845022 -> 1788845022 00000004:00010000:2.0:1788845023.123878:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845023.123913:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:2.0:1788845023.123922:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ad6e1f06 reply state 0000000030afac3d transno 4294967611 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.123932:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845023.124229:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641458560:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 1023us (2135us total) trans 4294967611 rc 0/0 00000100:00100000:1.0:1788845023.128530:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.129225:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641458688 00000100:00100000:3.0:1788845023.129239:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641458688 00002000:00010000:3.0:1788845023.129288:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1854529->18446744073709551615) 00002000:00100000:3.0:1788845023.129293:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be6300 with x1875739641458688 ext(1854529->18446744073709551615) 00002000:00010000:3.0:1788845023.129306:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be6300. 00000100:00100000:3.0:1788845023.129342:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 285 00000100:00100000:3.0:1788845023.129354:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641458688:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845023.129719:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1604682 -> 1854529 00002000:00010000:3.0:1788845023.129724:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00002000:00010000:3.0:1788845023.129727:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845021 -> 1788845022 00002000:00010000:3.0:1788845023.129729:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00002000:00010000:3.0:1788845023.129731:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 73 -> 97 00000100:00100000:3.0:1788845023.129805:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641458688:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 452us (1287us total) trans 4294967391 rc 0/0 00000100:00100000:3.0:1788845023.129829:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 285 00002000:00010000:3.0:1788845023.129833:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1854529->18446744073709551615) 00002000:00100000:3.0:1788845023.129835:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be6300 with x1875739641458688 ext(1854529->18446744073709551615) 00002000:00010000:3.0:1788845023.129840:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be6300. 00000100:00100000:2.0:1788845023.132378:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 765 00000020:00080000:2.0:1788845023.132392:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845023.137312:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.137332:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.137340:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.137346:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216fa00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845023.137352:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.137359:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.137397:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.137401:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845023.137407:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.137440:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac42b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.139607:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.139758:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641458816 00000100:00100000:3.0:1788845023.139768:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641458816 00002000:00100000:3.0:1788845023.139827:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1605632->1855487) req@ffff949f023dd880 x1875739641458816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.139838:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dd880 with x1875739641458816 ext(1605632->1855487) 00002000:00010000:3.0:1788845023.139844:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dd880 00000100:00100000:3.0:1788845023.139878:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 286 00000100:00100000:3.0:1788845023.139917:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641458816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.146751:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641458816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 6825us (7144us total) trans 0 rc 249856/249856 00000100:00100000:2.0:1788845023.146765:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 286 00002000:00100000:2.0:1788845023.146778:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1605632->1855487) req@ffff949f023dd880 x1875739641458816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/440 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 249856/249856 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.146789:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dd880 with x1875739641458816 ext(1605632->1855487) 00002000:00010000:2.0:1788845023.146795:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dd880 00000100:00100000:0.0:1788845023.152611:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.152715:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641458944 00000100:00100000:1.0:1788845023.152725:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641458944 00002000:00100000:1.0:1788845023.152776:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1835008->1854528) req@ffff94a035447480 x1875739641458944/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845023.152787:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035447480 with x1875739641458944 ext(1835008->1854528) 00002000:00010000:1.0:1788845023.152793:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035447480 00000100:00100000:1.0:1788845023.152828:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 287 00000100:00100000:1.0:1788845023.152839:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035447480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641458944:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845023.156803:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035447480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641458944:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3954us (4191us total) trans 4294967392 rc 0/0 00000100:00100000:1.0:1788845023.156817:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 287 00002000:00100000:1.0:1788845023.156830:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1835008->1854528) req@ffff94a035447480 x1875739641458944/t4294967392(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/448 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845023.156841:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035447480 with x1875739641458944 ext(1835008->1854528) 00002000:00010000:1.0:1788845023.156846:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035447480 00000100:00100000:0.0:1788845023.159999:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.160223:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641459072 00000100:00100000:1.0:1788845023.160251:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641459072 00000100:00100000:1.0:1788845023.160314:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 496 00000100:00100000:1.0:1788845023.160324:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641459072:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:0.0:1788845023.218855:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967392 is committed 00000100:00100000:1.0:1788845023.219109:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641459072:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 58781us (59116us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.219123:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 496 00000100:00100000:3.0:1788845023.222007:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.222267:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641459200 00000100:00100000:0.0:1788845023.222278:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641459200 00000100:00100000:0.0:1788845023.222349:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 766 00000100:00100000:0.0:1788845023.222359:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641459200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.222382:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.222403:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac432 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7dba expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.222529:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ac439 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.222550:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ac439 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.222561:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ac439 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845023.222589:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1854529 00000004:00200000:0.0:1788845023.222593:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845023.222610:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ac439 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.222658:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac432 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dba expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.222692:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ac439 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dba expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.222700:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.222703:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845023.222706:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbac00, rc 0) 00000100:00100000:0.0:1788845023.222770:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641459200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 412us (774us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.222778:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 766 00000100:00100000:0.0:1788845023.225829:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.225874:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.226007:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641459328 00000100:00100000:3.0:1788845023.226026:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641459328 00000100:00100000:3.0:1788845023.226083:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 767 00000100:00100000:3.0:1788845023.226094:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641459328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845023.226115:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845023.226135:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac440 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7dc1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.226173:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.226213:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660206720:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845023.226309:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.227757:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641459456 00000100:00100000:1.0:1788845023.227767:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641459456 00000100:00100000:1.0:1788845023.227826:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 497 00000100:00100000:1.0:1788845023.227836:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641459456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.227852:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845023.227867:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ac44c572/0xb2187301287ac447 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7dc8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.227952:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660206848:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845023.228031:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788845023.229452:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845022 -> 1788845023 00000100:00100000:3.0:1788845023.229482:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660206720:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845023.229502:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:3.0:1788845023.229518:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac440 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dc1 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.229548:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845023.229551:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845023.229556:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac440 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dc1 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.229571:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845023.229628:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641459328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3536us (3809us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845023.229637:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 767 00002000:00010000:3.0:1788845023.232362:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845022 -> 1788845023 00002000:00010000:3.0:1788845023.232373:0:7983:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 97 -> 137 00000100:00100000:3.0:1788845023.232398:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660206848:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845023.232436:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ac44c572/0xb2187301287ac447 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dc8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.232458:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ac44c572/0xb2187301287ac447 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dc8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.232470:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ae00, rc 0) 00000100:00100000:3.0:1788845023.232630:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641459456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4794us (6753us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845023.232638:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 497 00000100:00100000:1.0:1788845023.243197:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.243278:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.243352:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641459712 00000100:00100000:2.0:1788845023.243363:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641459712 00000100:00100000:2.0:1788845023.243438:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 498 00000100:00100000:2.0:1788845023.243448:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641459712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.243469:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.243487:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ac44e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7dd6 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.243556:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02a48e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660206976:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.243667:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845023.246595:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02a48e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660206976:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845023.246640:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ac44e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dd6 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.246667:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ac44e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dd6 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.246683:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845023.246760:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641459712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3312us (3480us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845023.246769:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 498 00000100:00100000:0.0:1788845023.251659:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641459584 00000100:00100000:0.0:1788845023.251677:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641459584 00000100:00100000:1.0:1788845023.252173:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 768 00000100:00100000:1.0:1788845023.252190:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641459584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.252217:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845023.252243:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac455 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7dcf expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.252485:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.252614:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660207104:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845023.252747:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.257963:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660207104:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845023.257994:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845023.258025:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac455 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dcf expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.258047:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.258050:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.258056:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac455 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dcf expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.258072:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1be00, rc 0) 00000100:00100000:1.0:1788845023.258150:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641459584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5965us (14962us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845023.258158:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 768 00000100:00100000:3.0:1788845023.265649:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.265835:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641459840 00000100:00100000:1.0:1788845023.265845:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641459840 00000100:00100000:1.0:1788845023.265939:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 769 00000100:00100000:1.0:1788845023.265949:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641459840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.265973:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845023.265992:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac45c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ddd expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.266087:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.266115:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.266126:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845023.266155:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1854529 00000004:00200000:1.0:1788845023.266159:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845023.266177:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.266203:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ac45c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.266219:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.266228:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.266231:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845023.266235:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b800, rc 0) 00000100:00100000:1.0:1788845023.266296:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641459840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 348us (658us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.266304:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 769 00000100:00100000:3.0:1788845023.278504:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.278553:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.278720:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641459968 00000100:00100000:0.0:1788845023.278731:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641459968 00000100:00100000:0.0:1788845023.278806:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 770 00000100:00100000:0.0:1788845023.278816:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641459968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.278837:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.278858:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac46a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7de4 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.278933:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.278997:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660207232:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845023.279079:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.280162:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641460096 00000100:00100000:1.0:1788845023.280170:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641460096 00000100:00100000:0.0:1788845023.280339:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 499 00000100:00100000:0.0:1788845023.280348:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641460096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.280364:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.280378:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ac471 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7deb expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.280435:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f7adf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660207360:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845023.280538:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.284247:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7adf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660207360:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845023.284286:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ac471 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7deb expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.284328:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ac471 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7deb expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.284343:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba800, rc 0) 00000100:00100000:1.0:1788845023.284436:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641460096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4087us (5881us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845023.284445:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 499 00000100:00100000:0.0:1788845023.284595:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660207232:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845023.284609:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:0.0:1788845023.284621:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac46a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7de4 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.284633:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.284636:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845023.284641:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ac46a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7de4 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.284652:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba400, rc 0) 00000100:00100000:0.0:1788845023.284672:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641459968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5859us (6179us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845023.284678:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 770 00000100:00100000:2.0:1788845023.289630:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.289681:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.289749:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641460224 00000100:00100000:3.0:1788845023.289758:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641460224 00000100:00100000:3.0:1788845023.289820:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 771 00000100:00100000:3.0:1788845023.289829:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641460224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845023.289848:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845023.289867:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac478 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7df2 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.289928:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.289964:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660207488:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845023.290073:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641460352 00000100:00100000:0.0:1788845023.290078:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641460352 00000100:00100000:0.0:1788845023.290110:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 500 00000100:00100000:0.0:1788845023.290117:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641460352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.290126:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.290137:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ac47f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7df9 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.290171:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660207616:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845023.291040:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.291099:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845023.305389:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660207488:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845023.305415:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:3.0:1788845023.305436:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac478 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7df2 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.305469:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845023.305473:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845023.305479:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ac478 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7df2 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.305511:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845023.305582:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641460224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15753us (15962us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845023.305591:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 771 00000100:00100000:0.0:1788845023.306248:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660207616:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845023.306285:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ac47f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7df9 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.306302:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ac47f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7df9 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.306315:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb000, rc 0) 00000100:00100000:0.0:1788845023.306390:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641460352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16273us (16709us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845023.306397:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 500 00000100:00100000:2.0:1788845023.326677:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.326789:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641460480 00000100:00100000:0.0:1788845023.326800:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641460480 00002000:00100000:0.0:1788845023.326856:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346faa00 x1875739641460480/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845023.326867:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346faa00 with x1875739641460480 00000100:00100000:0.0:1788845023.326947:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 470 00000100:00100000:0.0:1788845023.326957:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641460480:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845023.344355:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641460480:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17386us (17677us total) trans 0 rc 1007616/1007616 00000100:00100000:0.0:1788845023.344370:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 470 00002000:00100000:0.0:1788845023.344386:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346faa00 x1875739641460480/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:129/0 lens 488/440 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 1007616/1007616 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845023.344397:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346faa00 with x1875739641460480 00000100:00100000:1.0:1788845023.352426:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.352505:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641460608 00000100:00100000:3.0:1788845023.352516:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641460608 00000100:00100000:3.0:1788845023.352590:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 501 00000100:00100000:3.0:1788845023.352602:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641460608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845023.352622:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845023.352642:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e00 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.352677:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.352686:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:3.0:1788845023.352701:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:3.0:1788845023.352708:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1093632->1155071] (req 1093632->1155071) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.352746:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.352765:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 5 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.352816:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660207744:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845023.353039:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.365229:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.365336:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641460736 00000100:00100000:0.0:1788845023.365346:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641460736 00010000:00010000:0.0:1788845023.365395:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 6 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.365443:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 282 00000100:00100000:0.0:1788845023.365454:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641460736:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845023.365481:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845023.365506:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 6 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.365525:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 6 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.365549:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788845023.365554:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788845023.365563:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.365588:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.365628:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.365643:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660207872:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845023.365736:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00010000:00010000:2.0:1788845023.366637:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ac416 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dac expref: 6 pid: 6682 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.366657:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660207744:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845023.366682:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.366694:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:2.0:1788845023.366791:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641460608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 14191us (14378us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845023.366808:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 501 00000100:00100000:0.0:1788845023.380253:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660207872:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845023.380286:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845023.380364:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641460736:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 14911us (15144us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.380377:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 282 00000100:00100000:3.0:1788845023.382101:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.384075:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641460864 00000100:00100000:2.0:1788845023.384088:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641460864 00000100:00100000:2.0:1788845023.384169:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 288 00000100:00100000:2.0:1788845023.384179:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641460864:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:2.0:1788845023.384489:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1854529 -> 1854529 00002000:00010000:2.0:1788845023.384503:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00002000:00010000:2.0:1788845023.384506:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845023 -> 1788845022 00002000:00010000:2.0:1788845023.384509:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00002000:00010000:2.0:1788845023.384511:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 137 -> 137 00000100:00100000:2.0:1788845023.384586:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641460864:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 408us (2494us total) trans 4294967393 rc 0/0 00000100:00100000:2.0:1788845023.384595:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 288 00000100:00100000:3.0:1788845023.397018:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.399390:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641460992 00000100:00100000:2.0:1788845023.399407:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641460992 00000100:00100000:2.0:1788845023.399489:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 772 00000100:00100000:2.0:1788845023.399510:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641460992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.399533:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.399563:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ac48d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e07 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.399602:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.399649:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660208000:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.399763:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845023.405255:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660208000:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845023.405291:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:2.0:1788845023.405315:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ac48d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e07 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.405339:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845023.405342:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845023.405349:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ac48d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e07 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.405366:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:2.0:1788845023.405439:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641460992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5930us (8439us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845023.405448:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 772 00000100:00100000:3.0:1788845023.427768:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.427963:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641461120 00000100:00100000:0.0:1788845023.427974:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641461120 00000100:00100000:0.0:1788845023.428384:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 773 00000100:00100000:0.0:1788845023.428427:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641461120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.428449:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.428470:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac494 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e0e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.428522:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.428699:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660208128:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845023.428801:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.440693:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660208128:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845023.440725:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:0.0:1788845023.440746:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac494 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e0e expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.440770:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.440778:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845023.440784:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac494 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e0e expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.440804:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:0.0:1788845023.441368:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641461120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12972us (13608us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845023.441377:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 773 00000100:00100000:2.0:1788845023.464879:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.465022:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641461248 00000100:00100000:0.0:1788845023.465033:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641461248 00000100:00100000:0.0:1788845023.465097:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 774 00000100:00100000:0.0:1788845023.465109:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641461248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.465130:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.465151:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac49b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e15 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.465188:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.465243:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660208256:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845023.465316:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.468264:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f7af800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660208256:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845023.468294:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:0.0:1788845023.468314:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac49b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e15 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.474694:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.474699:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845023.474714:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac49b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e15 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.474739:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:0.0:1788845023.475036:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641461248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9927us (10166us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845023.475048:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 774 00000100:00100000:1.0:1788845023.505386:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.507633:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641461376 00000100:00100000:3.0:1788845023.507649:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641461376 00000100:00100000:3.0:1788845023.507856:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 775 00000100:00100000:3.0:1788845023.507868:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641461376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845023.507943:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845023.507968:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac4a2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e1c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.508023:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.508077:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660208384:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845023.508178:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.511596:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660208384:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845023.511630:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845023.511652:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac4a2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e1c expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.511673:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.511677:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.511683:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ac4a2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e1c expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.511710:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845023.511785:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641461376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3918us (6408us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845023.511794:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 775 00000100:00100000:1.0:1788845023.520268:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.520579:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641461504 00000100:00100000:2.0:1788845023.520592:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641461504 00002000:00100000:2.0:1788845023.520648:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff94a007eb5500 x1875739641461504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.520661:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb5500 with x1875739641461504 ext(1245184->1249279) 00002000:00010000:2.0:1788845023.520667:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb5500 00000100:00100000:2.0:1788845023.520710:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 289 00000100:00100000:2.0:1788845023.520720:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641461504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.521294:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641461504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 574us (1035us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.521319:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 289 00002000:00100000:2.0:1788845023.521328:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff94a007eb5500 x1875739641461504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.521337:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb5500 with x1875739641461504 ext(1245184->1249279) 00002000:00010000:2.0:1788845023.521341:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb5500 00000100:00100000:2.0:1788845023.547469:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.547614:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641461632 00000100:00100000:3.0:1788845023.547628:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641461632 00002000:00100000:3.0:1788845023.547699:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1253375) req@ffff949f036cc700 x1875739641461632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.547711:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f036cc700 with x1875739641461632 ext(1249280->1253375) 00002000:00010000:3.0:1788845023.547717:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f036cc700 00000100:00100000:3.0:1788845023.547751:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 290 00000100:00100000:3.0:1788845023.547790:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641461632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845023.548126:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641461632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 338us (667us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845023.548134:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 290 00002000:00100000:3.0:1788845023.548140:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1253375) req@ffff949f036cc700 x1875739641461632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.548149:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f036cc700 with x1875739641461632 ext(1249280->1253375) 00002000:00010000:3.0:1788845023.548153:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f036cc700 00000100:00100000:0.0:1788845023.553734:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.553820:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641461760 00000100:00100000:3.0:1788845023.553830:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641461760 00002000:00100000:3.0:1788845023.553925:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1257471) req@ffff94a03f7af800 x1875739641461760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.553938:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f7af800 with x1875739641461760 ext(1253376->1257471) 00002000:00010000:3.0:1788845023.553944:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f7af800 00000100:00100000:3.0:1788845023.553981:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 291 00000100:00100000:3.0:1788845023.553991:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f7af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641461760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845023.554248:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f7af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641461760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 257us (525us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845023.554256:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 291 00002000:00100000:3.0:1788845023.554263:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1257471) req@ffff94a03f7af800 x1875739641461760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.554272:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f7af800 with x1875739641461760 ext(1253376->1257471) 00002000:00010000:3.0:1788845023.554277:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f7af800 00000100:00100000:1.0:1788845023.560770:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.561265:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641461888 00000100:00100000:2.0:1788845023.561277:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641461888 00002000:00100000:2.0:1788845023.561333:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1257472->1261567) req@ffff94a007eb4380 x1875739641461888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.561346:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb4380 with x1875739641461888 ext(1257472->1261567) 00002000:00010000:2.0:1788845023.561352:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb4380 00000100:00100000:2.0:1788845023.561426:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 292 00000100:00100000:2.0:1788845023.561437:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641461888:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.561743:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641461888:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 307us (981us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.561751:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 292 00002000:00100000:2.0:1788845023.561759:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1257472->1261567) req@ffff94a007eb4380 x1875739641461888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.561768:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb4380 with x1875739641461888 ext(1257472->1261567) 00002000:00010000:2.0:1788845023.561772:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb4380 00000100:00100000:1.0:1788845023.565291:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.565392:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641462016 00000100:00100000:2.0:1788845023.565403:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641462016 00002000:00100000:2.0:1788845023.565474:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1261568->1265663) req@ffff94a007eb4e00 x1875739641462016/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.565488:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb4e00 with x1875739641462016 ext(1261568->1265663) 00002000:00010000:2.0:1788845023.565495:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb4e00 00000100:00100000:2.0:1788845023.565565:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 293 00000100:00100000:2.0:1788845023.565575:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a007eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641462016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.565841:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a007eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641462016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (559us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.565850:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 293 00002000:00100000:2.0:1788845023.565858:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1261568->1265663) req@ffff94a007eb4e00 x1875739641462016/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.565867:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a007eb4e00 with x1875739641462016 ext(1261568->1265663) 00002000:00010000:2.0:1788845023.565871:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a007eb4e00 00000100:00100000:0.0:1788845023.574404:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.574684:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641462144 00000100:00100000:2.0:1788845023.574694:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641462144 00002000:00100000:2.0:1788845023.574747:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1265664->1269759) req@ffff94a03f7ace00 x1875739641462144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.574759:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f7ace00 with x1875739641462144 ext(1265664->1269759) 00002000:00010000:2.0:1788845023.574765:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f7ace00 00000100:00100000:2.0:1788845023.574806:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 294 00000100:00100000:2.0:1788845023.574816:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f7ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641462144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.575367:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f7ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641462144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 549us (971us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.575377:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 294 00002000:00100000:2.0:1788845023.575401:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1265664->1269759) req@ffff94a03f7ace00 x1875739641462144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.575410:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f7ace00 with x1875739641462144 ext(1265664->1269759) 00002000:00010000:2.0:1788845023.575414:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f7ace00 00000100:00100000:1.0:1788845023.588115:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.588320:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641462272 00000100:00100000:2.0:1788845023.588330:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641462272 00002000:00100000:2.0:1788845023.588382:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1269760->1273855) req@ffff94a0360e9c00 x1875739641462272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.588393:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360e9c00 with x1875739641462272 ext(1269760->1273855) 00002000:00010000:2.0:1788845023.588398:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360e9c00 00000100:00100000:2.0:1788845023.588429:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 295 00000100:00100000:2.0:1788845023.588437:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641462272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.588688:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641462272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (580us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.588697:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 295 00002000:00100000:2.0:1788845023.588704:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1269760->1273855) req@ffff94a0360e9c00 x1875739641462272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.588712:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360e9c00 with x1875739641462272 ext(1269760->1273855) 00002000:00010000:2.0:1788845023.588715:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360e9c00 00000100:00100000:1.0:1788845023.598383:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.598564:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641462400 00000100:00100000:2.0:1788845023.598574:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641462400 00002000:00100000:2.0:1788845023.598625:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a0360eaa00 x1875739641462400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.598637:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360eaa00 with x1875739641462400 ext(1273856->1277951) 00002000:00010000:2.0:1788845023.598658:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360eaa00 00000100:00100000:2.0:1788845023.598707:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 296 00000100:00100000:2.0:1788845023.598718:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641462400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.599254:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641462400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 536us (880us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.599263:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 296 00002000:00100000:2.0:1788845023.599270:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a0360eaa00 x1875739641462400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.599280:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360eaa00 with x1875739641462400 ext(1273856->1277951) 00002000:00010000:2.0:1788845023.599284:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360eaa00 00000100:00100000:0.0:1788845023.606220:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.609686:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641462528 00000100:00100000:3.0:1788845023.609712:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641462528 00002000:00100000:3.0:1788845023.609789:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1277952->1282047) req@ffff94a03f7adf80 x1875739641462528/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.609802:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f7adf80 with x1875739641462528 ext(1277952->1282047) 00002000:00010000:3.0:1788845023.609809:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f7adf80 00000100:00100000:3.0:1788845023.609857:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 297 00000100:00100000:3.0:1788845023.609867:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f7adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641462528:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845023.610170:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f7adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641462528:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 304us (3959us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845023.610178:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 297 00002000:00100000:3.0:1788845023.610185:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1277952->1282047) req@ffff94a03f7adf80 x1875739641462528/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.610194:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f7adf80 with x1875739641462528 ext(1277952->1282047) 00002000:00010000:3.0:1788845023.610197:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f7adf80 00000100:00100000:1.0:1788845023.614612:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.614714:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641462656 00000100:00100000:2.0:1788845023.614724:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641462656 00002000:00100000:2.0:1788845023.614772:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1282048->1286143) req@ffff94a0360e8e00 x1875739641462656/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.614793:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360e8e00 with x1875739641462656 ext(1282048->1286143) 00002000:00010000:2.0:1788845023.614800:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360e8e00 00000100:00100000:2.0:1788845023.614837:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 298 00000100:00100000:2.0:1788845023.614848:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641462656:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.615148:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641462656:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 300us (544us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.615156:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 298 00002000:00100000:2.0:1788845023.615163:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1282048->1286143) req@ffff94a0360e8e00 x1875739641462656/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.615172:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360e8e00 with x1875739641462656 ext(1282048->1286143) 00002000:00010000:2.0:1788845023.615183:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360e8e00 00000100:00100000:1.0:1788845023.622035:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.622554:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641462784 00000100:00100000:2.0:1788845023.622565:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641462784 00002000:00100000:2.0:1788845023.622624:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1286144->1290239) req@ffff94a0360ea300 x1875739641462784/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.622636:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ea300 with x1875739641462784 ext(1286144->1290239) 00002000:00010000:2.0:1788845023.622642:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ea300 00000100:00100000:2.0:1788845023.622677:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 299 00000100:00100000:2.0:1788845023.622688:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641462784:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.623207:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641462784:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 520us (1182us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.623216:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 299 00002000:00100000:2.0:1788845023.623223:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1286144->1290239) req@ffff94a0360ea300 x1875739641462784/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.623231:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ea300 with x1875739641462784 ext(1286144->1290239) 00002000:00010000:2.0:1788845023.623235:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ea300 00000100:00100000:0.0:1788845023.628852:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.629291:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641462912 00000100:00100000:3.0:1788845023.629304:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641462912 00002000:00100000:3.0:1788845023.629355:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1290240->1294335) req@ffff94a03f6c0700 x1875739641462912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.629368:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c0700 with x1875739641462912 ext(1290240->1294335) 00002000:00010000:3.0:1788845023.629374:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c0700 00000100:00100000:3.0:1788845023.629417:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 300 00000100:00100000:3.0:1788845023.629427:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641462912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845023.629665:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641462912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 238us (823us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845023.629687:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 300 00002000:00100000:3.0:1788845023.629695:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1290240->1294335) req@ffff94a03f6c0700 x1875739641462912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845023.629703:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c0700 with x1875739641462912 ext(1290240->1294335) 00002000:00010000:3.0:1788845023.629707:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c0700 00000100:00100000:3.0:1788845023.637650:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.638120:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641463040 00000100:00100000:1.0:1788845023.638130:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641463040 00002000:00100000:2.0:1788845023.638258:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1294336->1298431) req@ffff94a03f6c7480 x1875739641463040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.638267:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c7480 with x1875739641463040 ext(1294336->1298431) 00002000:00010000:2.0:1788845023.638272:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c7480 00000100:00100000:2.0:1788845023.638298:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 301 00000100:00100000:2.0:1788845023.638308:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641463040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845023.638505:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641463040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 198us (865us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845023.638539:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 301 00002000:00100000:2.0:1788845023.638547:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1294336->1298431) req@ffff94a03f6c7480 x1875739641463040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/4536 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.638555:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c7480 with x1875739641463040 ext(1294336->1298431) 00002000:00010000:2.0:1788845023.638558:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c7480 00000100:00100000:1.0:1788845023.656386:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.657103:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641463168 00000100:00100000:3.0:1788845023.657115:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641463168 00000100:00100000:3.0:1788845023.657183:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 502 00000100:00100000:3.0:1788845023.657193:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641463168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845023.657212:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845023.657230:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ac4a9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e23 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.657300:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660208512:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845023.657396:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ed80 going to sleep for 11 seconds 00002000:00010000:3.0:1788845023.677184:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845022 -> 1788845023 00000100:00100000:3.0:1788845023.677218:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660208512:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845023.677253:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ac4a9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e23 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.677276:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ac4a9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e23 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.677312:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:3.0:1788845023.677390:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641463168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 20198us (21015us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845023.677400:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 502 00000100:00100000:1.0:1788845023.687871:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.698163:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641463296 00000100:00100000:0.0:1788845023.698178:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641463296 00000100:00100000:0.0:1788845023.698267:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 503 00000100:00100000:0.0:1788845023.698278:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641463296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.698298:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.698334:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cb10f58/0xb2187301287ac4b0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e2a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.698396:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660208640:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845023.698498:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.702288:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660208640:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845023.702334:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cb10f58/0xb2187301287ac4b0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e2a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.702360:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cb10f58/0xb2187301287ac4b0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e2a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.702376:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbae00, rc 0) 00000100:00100000:0.0:1788845023.702442:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641463296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4165us (14581us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845023.702452:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 503 00000100:00100000:3.0:1788845023.706520:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.706656:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641463424 00000100:00100000:2.0:1788845023.706665:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641463424 00000100:00100000:2.0:1788845023.706745:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 776 00000100:00100000:2.0:1788845023.706755:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641463424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.706777:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.706800:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e31 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.706831:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.706842:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.706880:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.706937:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 15 pid: 5840 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.706952:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f036cca80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660208768:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845023.707074:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.712765:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f036cca80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660208768:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845023.712805:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.712826:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.712829:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.712833:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.712844:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:1.0:1788845023.712953:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641463424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6199us (6444us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.712963:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 776 00000100:00100000:1.0:1788845023.724726:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.724913:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641463552 00000100:00100000:0.0:1788845023.724924:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641463552 00010000:00010000:0.0:1788845023.724969:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 16 pid: 5840 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.725012:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 283 00000100:00100000:0.0:1788845023.725022:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641463552:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845023.725038:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845023.725063:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 16 pid: 5840 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.725098:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 16 pid: 5840 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.725119:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac40f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7da5 expref: 16 pid: 5840 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.725136:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.725142:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.725153:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.725162:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.725194:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.725207:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660208896:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845023.725290:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.729070:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660208896:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845023.729100:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845023.729149:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641463552:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4129us (4432us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.729175:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 283 00000100:00100000:2.0:1788845023.743847:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.754574:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641463680 00000100:00100000:3.0:1788845023.754592:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641463680 00000100:00100000:3.0:1788845023.754681:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:3.0:1788845023.754699:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f036cca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641463680:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845023.754916:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1007616 -> 1007616 00000004:00010000:3.0:1788845023.754920:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00000004:00010000:3.0:1788845023.754922:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:3.0:1788845023.754925:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00000004:00010000:3.0:1788845023.754927:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845023.755015:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f036cca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641463680:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 308us (11168us total) trans 4294967612 rc 0/0 00000100:00100000:3.0:1788845023.755023:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:1.0:1788845023.759593:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.759705:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641463808 00000100:00100000:0.0:1788845023.759715:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641463808 00000100:00100000:0.0:1788845023.759792:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 777 00000100:00100000:0.0:1788845023.759803:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641463808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.759839:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.759864:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbae00/0xb2187301287ac4be lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e38 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.759948:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.759990:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660209024:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845023.760075:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.765989:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.766059:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641463936 00000100:00100000:2.0:1788845023.766069:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641463936 00000100:00100000:2.0:1788845023.766135:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 504 00000100:00100000:2.0:1788845023.766146:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641463936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.766166:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.766184:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac4c5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e3f expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.766245:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660209024:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.766249:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660209152:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845023.766285:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845023.766302:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbae00/0xb2187301287ac4be lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e38 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.766320:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.766323:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.766327:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbae00/0xb2187301287ac4be lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e38 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.766337:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1b40 going to sleep for 11 seconds 00010000:00010000:1.0:1788845023.766346:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbae00, rc 0) 00000100:00100000:1.0:1788845023.766381:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641463808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6579us (6799us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845023.766389:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 777 00000100:00100000:3.0:1788845023.780559:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660209152:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845023.780606:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac4c5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e3f expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.780632:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac4c5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e3f expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.780670:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:3.0:1788845023.780753:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641463936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 14609us (14773us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845023.780762:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 504 00000100:00100000:0.0:1788845023.786769:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.786819:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.786853:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641464064 00000100:00100000:1.0:1788845023.786864:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641464064 00000100:00100000:1.0:1788845023.786963:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 778 00000100:00100000:1.0:1788845023.786976:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641464064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.787001:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1788845023.787024:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641464192 00000100:00100000:2.0:1788845023.787029:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641464192 00010000:00010000:1.0:1788845023.787033:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4cc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e46 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.787065:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 505 00000100:00100000:2.0:1788845023.787073:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641464192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.787073:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.787081:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.787091:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac4d3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e4d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.787117:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660209280:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.787124:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660209408:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.788185:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845023.791185:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660209408:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845023.791232:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac4d3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e4d expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.791281:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac4d3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e4d expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.791298:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845023.791369:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641464192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4297us (4548us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845023.791378:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 505 00000100:00100000:1.0:1788845023.791684:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.794504:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660209280:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845023.794617:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845023.794637:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4cc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e46 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.794657:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.794660:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.794665:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4cc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e46 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.794679:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2800, rc 0) 00000100:00100000:1.0:1788845023.795599:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641464064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8620us (8834us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845023.795610:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 778 00000100:00100000:3.0:1788845023.805030:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.805075:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.805162:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641464320 00000100:00100000:2.0:1788845023.805171:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641464320 00000100:00100000:2.0:1788845023.805224:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 779 00000100:00100000:2.0:1788845023.805233:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641464320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.805252:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.805272:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac4da lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e54 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.805309:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.805347:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660209536:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.805421:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.805681:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641464448 00000100:00100000:1.0:1788845023.805686:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641464448 00000100:00100000:1.0:1788845023.805727:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 506 00000100:00100000:1.0:1788845023.805735:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641464448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.805744:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845023.805755:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b52f3c29/0xb2187301287ac4e1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e5b expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.805791:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660209664:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845023.805843:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.809854:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660209664:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.809867:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660209536:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845023.809910:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:2.0:1788845023.809929:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac4da lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e54 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.809930:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b52f3c29/0xb2187301287ac4e1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e5b expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.809971:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.809973:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b52f3c29/0xb2187301287ac4e1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e5b expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.809974:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845023.809978:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ac4da lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e54 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.809986:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2800, rc 0) 00010000:00010000:2.0:1788845023.809988:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845023.810061:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641464320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4828us (4984us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845023.810070:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 779 00000100:00100000:1.0:1788845023.814207:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641464448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8464us (9180us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845023.814220:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 506 00000100:00100000:2.0:1788845023.824758:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.824825:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.824978:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641464704 00000100:00100000:0.0:1788845023.824989:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641464704 00000100:00100000:0.0:1788845023.825059:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 507 00000100:00100000:0.0:1788845023.825069:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641464704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.825091:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.825111:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287ac4e8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e69 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.825181:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660209792:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845023.825296:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845023.825470:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641464576 00000100:00100000:3.0:1788845023.825475:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641464576 00000100:00100000:3.0:1788845023.825519:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 780 00000100:00100000:3.0:1788845023.825527:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641464576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845023.825539:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845023.825554:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ac4ef lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e62 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.825585:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.825609:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660209920:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845023.825638:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ed80 going to sleep for 11 seconds 00010000:00010000:3.0:1788845023.826199:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845023.831404:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660209792:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845023.831448:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287ac4e8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e69 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.831473:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287ac4e8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e69 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.831490:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb600, rc 0) 00000100:00100000:0.0:1788845023.831558:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641464704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6490us (6811us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845023.831567:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 507 00000100:00100000:1.0:1788845023.831698:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660209920:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845023.831713:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1007616 00010000:00010000:1.0:1788845023.831725:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ac4ef lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e62 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.831738:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.831741:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.831746:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ac4ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e62 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.831757:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845023.831779:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641464576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6254us (6955us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845023.831787:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 780 00000100:00100000:1.0:1788845023.835410:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.836189:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641464960 00000100:00100000:2.0:1788845023.836200:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641464960 00000100:00100000:1.0:1788845023.836387:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 781 00000100:00100000:1.0:1788845023.836401:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641464960:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845023.836472:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845023.836497:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ac439 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7dba expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.836529:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.836534:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845023.837477:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.837499:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.837509:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.837613:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.837646:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 14 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.837662:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660210048:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845023.838413:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.844716:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660210048:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845023.844757:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.845120:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.845353:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641465088 00000100:00100000:1.0:1788845023.845373:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641465088 00010000:00010000:1.0:1788845023.845404:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 15 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.845438:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 284 00000100:00100000:1.0:1788845023.845445:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641465088:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845023.845456:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845023.845461:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 15 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.845474:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 15 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.845493:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ac463 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ddd expref: 15 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.845505:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.845508:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.845515:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.845520:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.845551:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845023.845607:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641465088:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 161us (496us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.845624:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 284 00010000:00010000:2.0:1788845023.845626:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.845633:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845023.847241:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967613 00000004:00010000:2.0:1788845023.848662:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1007616 -> 1007616 00000004:00010000:2.0:1788845023.848670:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845023.848673:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845023.848675:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845023.848677:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:2.0:1788845023.848690:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001107f4d0 reply state 00000000c62de721 transno 4294967614 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.848715:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845023.849685:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641464960:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13270us (14275us total) trans 4294967614 rc 0/0 00000100:00100000:2.0:1788845023.849695:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 781 00000020:00080000:2.0:1788845023.849702:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845023.853458:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.853476:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.853484:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.853490:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f024f2800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845023.853495:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.853503:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.853520:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.853627:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845023.853634:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.853642:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac4f6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.858484:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.858873:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641465344 00000100:00100000:3.0:1788845023.858927:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641465344 00002000:00010000:3.0:1788845023.858974:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845023.858979:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5880 with x1875739641465344 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845023.858992:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5880. 00000100:00100000:3.0:1788845023.859091:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 302 00000100:00100000:3.0:1788845023.859102:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641465344:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845023.859469:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1854529 -> 0 00002000:00010000:3.0:1788845023.859473:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00002000:00010000:3.0:1788845023.859475:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845023 -> 1788845022 00002000:00010000:3.0:1788845023.859477:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00002000:00010000:3.0:1788845023.859479:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 137 -> 137 00000100:00100000:3.0:1788845023.859859:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641465344:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 758us (1382us total) trans 4294967394 rc 0/0 00000100:00100000:3.0:1788845023.859866:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 302 00002000:00010000:3.0:1788845023.859870:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845023.859873:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5880 with x1875739641465344 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845023.859878:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5880. 00000100:00100000:2.0:1788845023.862049:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.864646:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641465216 00000100:00100000:3.0:1788845023.864663:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641465216 00002000:00010000:0.0:1788845023.867014:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845023.867024:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c4bb80 with x1875739641465216 00000100:00100000:0.0:1788845023.867079:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 472 00000100:00100000:0.0:1788845023.867091:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641465216:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845023.867393:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1007616 -> 766558 00000004:00010000:0.0:1788845023.867397:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845023.867400:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845023.867402:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845023.867405:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845023.867480:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641465216:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 391us (5440us total) trans 4294967615 rc 0/0 00000100:00100000:0.0:1788845023.867487:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 472 00002000:00010000:0.0:1788845023.867504:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845023.867506:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c4bb80 with x1875739641465216 00000100:00100000:2.0:1788845023.875368:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.875516:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641465472 00000100:00100000:0.0:1788845023.875527:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641465472 00000100:00100000:0.0:1788845023.875732:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 782 00000100:00100000:0.0:1788845023.875745:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641465472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.875767:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.875790:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac4fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e70 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.875938:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac504 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.875955:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.875966:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845023.875997:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 766558 00000004:00200000:0.0:1788845023.876011:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845023.876033:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.876061:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac4fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e70 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.876081:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac504 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e70 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.876108:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.876111:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845023.876115:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:0.0:1788845023.876181:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641465472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 439us (824us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.876188:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 782 00000100:00100000:2.0:1788845023.880446:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.880499:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.880767:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641465728 00000100:00100000:3.0:1788845023.880778:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641465728 00000100:00100000:3.0:1788845023.880842:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 508 00000100:00100000:3.0:1788845023.880853:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641465728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845023.880870:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845023.880922:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac50b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e7e expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.880984:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660210176:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845023.881074:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.881407:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641465600 00000100:00100000:1.0:1788845023.881411:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641465600 00000100:00100000:1.0:1788845023.881444:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 783 00000100:00100000:1.0:1788845023.881449:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641465600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.881460:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845023.881485:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac512 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e77 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.881510:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.881527:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660210304:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845023.881577:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00002000:00010000:3.0:1788845023.884875:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845022 -> 1788845023 00000100:00100000:3.0:1788845023.884973:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660210176:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845023.885021:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac50b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e7e expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.885046:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ac50b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e7e expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.885059:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:3.0:1788845023.885112:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641465728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4261us (4678us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845023.885122:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 508 00000100:00100000:1.0:1788845023.885418:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660210304:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845023.885431:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 766558 00010000:00010000:1.0:1788845023.885440:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac512 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e77 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.885450:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.885453:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.885456:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac512 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e77 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.885464:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2800, rc 0) 00000100:00100000:1.0:1788845023.885492:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641465600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4043us (4993us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845023.885497:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 783 00000100:00100000:2.0:1788845023.888558:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.888638:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641465856 00000100:00100000:1.0:1788845023.888648:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641465856 00000100:00100000:1.0:1788845023.888713:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 784 00000100:00100000:1.0:1788845023.888724:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641465856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.888745:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845023.888778:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac519 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e85 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.888815:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.888854:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660210432:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845023.888952:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.890089:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.890205:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641465984 00000100:00100000:2.0:1788845023.890212:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641465984 00000100:00100000:2.0:1788845023.890263:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 509 00000100:00100000:2.0:1788845023.890272:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641465984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.890286:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.890301:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac520 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e8c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.890347:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02482300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660210560:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.890414:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.894871:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660210432:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845023.895186:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 766558 00010000:00010000:1.0:1788845023.895209:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac519 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e85 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.895234:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.895237:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.895243:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac519 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e85 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.895269:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3c00, rc 0) 00000100:00100000:0.0:1788845023.895456:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641465856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6732us (6906us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845023.895468:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 784 00000100:00100000:2.0:1788845023.898717:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02482300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660210560:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845023.898761:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac520 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e8c expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.898811:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac520 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e8c expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.898828:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845023.899087:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641465984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8814us (9002us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845023.899097:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 509 00000100:00100000:3.0:1788845023.905143:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.905386:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641466240 00000100:00100000:2.0:1788845023.905396:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641466240 00000100:00100000:2.0:1788845023.905486:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 785 00000100:00100000:2.0:1788845023.905496:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641466240:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845023.905562:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845023.905592:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac504 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e70 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.905614:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845023.905619:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845023.905825:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.905839:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.905849:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845023.905963:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967616 00000004:00010000:2.0:1788845023.905988:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 766558 -> 766558 00000004:00010000:2.0:1788845023.905991:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845023.905994:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845023.905995:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845023.905997:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:2.0:1788845023.906017:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001e2846b1 reply state 00000000cbacad6a transno 4294967617 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.906027:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845023.906122:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641466240:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 626us (988us total) trans 4294967617 rc 0/0 00000100:00100000:2.0:1788845023.906129:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 785 00000020:00080000:2.0:1788845023.906134:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845023.907969:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845023.908027:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.908039:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.908045:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.908051:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216e800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845023.908056:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.908064:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.908079:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845023.908093:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845023.908098:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.908104:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ac527 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.908834:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641466368 00000100:00100000:1.0:1788845023.908845:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641466368 00000100:00100000:1.0:1788845023.908926:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 786 00000100:00100000:1.0:1788845023.908937:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641466368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845023.908961:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845023.908981:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e93 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.909021:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.909038:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.909071:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.909088:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 17 pid: 11036 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.909101:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660210688:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845023.909173:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845023.912177:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.912430:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660210688:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845023.912468:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.912486:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.912489:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845023.912493:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.912501:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3c00, rc 0) 00000100:00100000:2.0:1788845023.912661:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641466496 00000100:00100000:2.0:1788845023.912669:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641466496 00010000:00010000:2.0:1788845023.912704:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 18 pid: 11036 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.912744:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 285 00000100:00100000:2.0:1788845023.912755:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008917850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641466496:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845023.912768:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845023.912795:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 18 pid: 11036 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.912811:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 18 pid: 11036 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.912846:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ac4b7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e31 expref: 18 pid: 11036 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.912862:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845023.912867:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.912912:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.912921:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.912952:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.912965:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02482300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660210816:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845023.913088:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845023.915575:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02482300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660210816:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845023.915605:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845023.915667:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008917850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641466496:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2913us (3497us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845023.915678:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 285 00000100:00100000:0.0:1788845023.919089:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641466368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10141us (11115us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.919104:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 786 00000100:00100000:1.0:1788845023.922464:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.922548:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641466624 00000100:00100000:2.0:1788845023.922559:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641466624 00000100:00100000:2.0:1788845023.922639:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 510 00000100:00100000:2.0:1788845023.922650:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641466624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.922670:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.922688:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7e9a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.922721:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.922730:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(2 12) 00010000:00010000:2.0:1788845023.922734:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:2.0:1788845023.922739:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.922774:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.922792:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 7 pid: 7983 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.922806:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02482300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660210944:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845023.922945:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.926209:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.926315:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02482300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660210944:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845023.926359:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.926406:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845023.926471:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641466624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3822us (4016us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845023.926480:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 510 00000100:00100000:1.0:1788845023.927815:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641466752 00000100:00100000:1.0:1788845023.927829:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641466752 00010000:00010000:1.0:1788845023.927874:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 8 pid: 7983 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.927953:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 286 00000100:00100000:1.0:1788845023.927964:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641466752:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845023.927979:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845023.928018:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 8 pid: 7983 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.928038:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 8 pid: 7983 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.928063:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ac486 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1093632->1155071) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e00 expref: 8 pid: 7983 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.928079:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:1.0:1788845023.928084:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:1.0:1788845023.928093:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.928104:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.928143:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.928159:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660211072:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845023.928318:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845023.932145:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660211072:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845023.932176:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845023.932238:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641466752:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4276us (6036us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.932249:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 286 00000100:00100000:3.0:1788845023.934283:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.934328:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.934400:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641467008 00000100:00100000:2.0:1788845023.934410:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641467008 00002000:00010000:2.0:1788845023.934450:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845023.934455:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f23480 with x1875739641467008 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845023.934469:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f23480. 00000100:00100000:2.0:1788845023.934503:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 303 00000100:00100000:2.0:1788845023.934515:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641467008:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845023.934750:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1788845023.934754:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00002000:00010000:2.0:1788845023.934757:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845023 -> 1788845022 00002000:00010000:2.0:1788845023.934760:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00002000:00010000:2.0:1788845023.934762:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 137 -> 137 00000100:00100000:2.0:1788845023.934817:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641467008:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 305us (544us total) trans 4294967395 rc 0/0 00000100:00100000:2.0:1788845023.934824:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 303 00002000:00010000:2.0:1788845023.934828:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845023.934830:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f23480 with x1875739641467008 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845023.934834:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f23480. 00000100:00100000:0.0:1788845023.935212:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641466880 00000100:00100000:0.0:1788845023.935218:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641466880 00002000:00010000:0.0:1788845023.935243:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845023.935249:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20a80 with x1875739641466880 00000100:00100000:0.0:1788845023.935279:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 473 00000100:00100000:0.0:1788845023.935287:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641466880:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845023.935420:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 766558 -> 991973 00000004:00010000:0.0:1788845023.935423:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845023.935426:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845023.935428:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845023.935444:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845023.935486:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641466880:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 200us (1157us total) trans 4294967618 rc 0/0 00000100:00100000:0.0:1788845023.935492:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 473 00002000:00010000:0.0:1788845023.935495:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845023.935497:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20a80 with x1875739641466880 00000100:00100000:3.0:1788845023.938680:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.938851:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641467136 00000100:00100000:0.0:1788845023.938862:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641467136 00000100:00100000:0.0:1788845023.938959:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 787 00000100:00100000:0.0:1788845023.938969:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641467136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.938996:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.939029:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac53c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ea1 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.939116:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac543 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.939132:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac543 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.939143:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac543 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845023.939171:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 991973 00000004:00200000:0.0:1788845023.939175:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845023.939194:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac543 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.939220:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac53c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ea1 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.939261:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac543 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ea1 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.939270:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.939273:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845023.939276:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb600, rc 0) 00000100:00100000:0.0:1788845023.939391:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:14891:x1875739641467136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 422us (721us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.939398:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 787 00000100:00100000:2.0:1788845023.943855:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845023.944050:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641467264 00000100:00100000:3.0:1788845023.944060:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641467264 00000100:00100000:3.0:1788845023.944121:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 788 00000100:00100000:3.0:1788845023.944131:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641467264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845023.944155:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845023.944176:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ac54a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ea8 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845023.944210:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845023.944252:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660211200:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845023.944388:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845023.946276:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.948389:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660211200:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845023.948415:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:0.0:1788845023.948436:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ac54a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ea8 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.948458:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.948460:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845023.948465:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ac54a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ea8 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.948482:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:0.0:1788845023.948545:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641467264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4416us (4701us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845023.948554:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 788 00000100:00100000:2.0:1788845023.949133:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641467392 00000100:00100000:2.0:1788845023.949153:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641467392 00000100:00100000:2.0:1788845023.949197:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 511 00000100:00100000:2.0:1788845023.949205:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641467392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845023.949222:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845023.949238:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac551 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7eaf expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845023.949289:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660211328:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845023.949349:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1d80 going to sleep for 11 seconds 00002000:00010000:2.0:1788845023.952179:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845022 -> 1788845023 00000100:00100000:2.0:1788845023.952214:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660211328:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845023.952258:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac551 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eaf expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.952284:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac551 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eaf expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845023.952301:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845023.952363:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641467392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3158us (6096us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845023.952371:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 511 00000100:00100000:1.0:1788845023.956573:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845023.956661:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641467520 00000100:00100000:0.0:1788845023.956686:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641467520 00000100:00100000:0.0:1788845023.956753:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 789 00000100:00100000:0.0:1788845023.956764:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641467520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845023.956787:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845023.956810:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac558 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7eb6 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.956841:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.956852:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac558 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eb6 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.956927:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.956946:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 18 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.956959:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cb6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660211456:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845023.957054:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845023.960287:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845023.960389:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641467648 00000100:00100000:1.0:1788845023.960400:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641467648 00010000:00010000:1.0:1788845023.960443:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 19 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.960487:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 287 00000100:00100000:1.0:1788845023.960498:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641467648:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845023.960511:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845023.960550:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 19 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.960569:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 19 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.960593:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845023.960599:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac558 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eb6 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.960610:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac558 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eb6 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.960635:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac558 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eb6 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845023.960675:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac558 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eb6 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845023.960689:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660211584:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845023.960769:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00010000:00010000:0.0:1788845023.961054:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac52e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e93 expref: 19 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845023.961071:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cb6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660211456:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845023.961098:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac558 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eb6 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845023.961122:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845023.961125:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845023.961128:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:0.0:1788845023.961157:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641467520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4395us (4593us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845023.961164:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 789 00000100:00100000:1.0:1788845023.967516:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660211584:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845023.967545:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845023.967615:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641467648:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7118us (7339us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845023.967626:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 287 00000100:00100000:3.0:1788845023.970084:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845023.970218:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641467776 00000100:00100000:2.0:1788845023.970228:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641467776 00002000:00100000:2.0:1788845023.970281:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359ac000 x1875739641467776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/0 e 0 to 0 dl 1788845034 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845023.970293:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359ac000 with x1875739641467776 00000100:00100000:2.0:1788845023.970349:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 474 00000100:00100000:2.0:1788845023.970359:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641467776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845024.043583:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641467776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 73211us (73496us total) trans 0 rc 995328/995328 00000100:00100000:2.0:1788845024.043597:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 474 00002000:00100000:2.0:1788845024.043610:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359ac000 x1875739641467776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:129/0 lens 488/440 e 0 to 0 dl 1788845034 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.043621:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359ac000 with x1875739641467776 00000100:00100000:3.0:1788845024.049769:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.051753:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641467904 00000100:00100000:0.0:1788845024.051768:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641467904 00000100:00100000:0.0:1788845024.052215:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 790 00000100:00100000:0.0:1788845024.052227:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641467904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.052249:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.052270:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ebd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.052306:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845024.052319:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:0.0:1788845024.052328:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.052344:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.052348:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845024.052352:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e800, rc 0) 00000100:00100000:0.0:1788845024.052412:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2368:x1875739641467904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 188us (2652us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845024.052440:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 790 00000100:00100000:0.0:1788845024.065266:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845024.065350:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641468032 00000100:00100000:1.0:1788845024.065361:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641468032 00000100:00100000:1.0:1788845024.065421:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 791 00000100:00100000:1.0:1788845024.065431:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cb6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641468032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845024.065452:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845024.065471:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac566 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ec4 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.065569:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.065584:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.065614:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845024.065641:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 991973 00000004:00200000:1.0:1788845024.065646:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845024.065662:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.065831:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac566 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.065852:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.065861:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845024.065864:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845024.065868:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3800, rc 0) 00000100:00100000:1.0:1788845024.066085:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cb6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641468032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 655us (827us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845024.066092:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 791 00000100:00100000:1.0:1788845024.074665:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.074767:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641468160 00000100:00100000:3.0:1788845024.074779:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641468160 00000100:00100000:3.0:1788845024.074857:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 512 00000100:00100000:3.0:1788845024.074868:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641468160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845024.074926:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845024.074947:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ac574 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ecb expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845024.075017:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660211712:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845024.075103:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.078256:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660211712:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845024.078297:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ac574 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ecb expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.078322:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ac574 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ecb expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.078335:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:2.0:1788845024.078397:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641468160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3531us (3739us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.078405:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 512 00000100:00100000:0.0:1788845024.089551:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.089628:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641468288 00000100:00100000:2.0:1788845024.089637:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641468288 00000100:00100000:2.0:1788845024.089697:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 513 00000100:00100000:2.0:1788845024.089706:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641468288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.089725:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.089755:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ac57b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ed2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.089813:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660211840:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.089960:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.093601:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660211840:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845024.093648:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ac57b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed2 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.093722:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ac57b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed2 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.093740:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845024.093853:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641468288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4146us (4309us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.093862:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 513 00000100:00100000:2.0:1788845024.107702:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.107828:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641468416 00000100:00100000:3.0:1788845024.107839:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641468416 00000100:00100000:3.0:1788845024.107945:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 792 00000100:00100000:3.0:1788845024.107957:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641468416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845024.107980:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845024.107984:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845024.108039:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ac558 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eb6 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.108059:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845024.108064:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845024.108078:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ed9 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.108101:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.108110:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.108154:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.108168:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 18 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845024.108177:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660211968:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845024.108246:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845024.112263:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.112360:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641468544 00000100:00100000:0.0:1788845024.112370:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641468544 00010000:00010000:0.0:1788845024.112415:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 19 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.112457:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 288 00000100:00100000:0.0:1788845024.112468:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641468544:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845024.112482:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845024.112518:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 19 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.112536:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 19 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.112557:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.112563:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.112575:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.112584:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.112642:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.112657:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660212096:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845024.112735:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00010000:00010000:3.0:1788845024.114872:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ac55f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ebd expref: 19 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845024.115117:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660211968:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845024.115148:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.115160:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845024.115162:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845024.115164:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:3.0:1788845024.115211:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641468416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7258us (7520us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845024.115217:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 792 00000100:00100000:0.0:1788845024.115340:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660212096:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845024.115351:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845024.115366:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641468544:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2903us (3115us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845024.115372:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 288 00000100:00100000:2.0:1788845024.126213:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.126314:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641468672 00000100:00100000:0.0:1788845024.126324:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641468672 00000100:00100000:0.0:1788845024.126398:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 475 00000100:00100000:0.0:1788845024.126409:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641468672:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845024.126809:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991973 -> 991973 00000004:00010000:0.0:1788845024.126814:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845024.126816:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845024.126818:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845024.126820:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845024.127147:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641468672:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 738us (942us total) trans 4294967619 rc 0/0 00000100:00100000:0.0:1788845024.127155:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 475 00000100:00100000:1.0:1788845024.130952:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.131086:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641468800 00000100:00100000:0.0:1788845024.131096:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641468800 00000100:00100000:0.0:1788845024.131170:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 793 00000100:00100000:0.0:1788845024.131179:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641468800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.131199:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.131218:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ac589 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ee0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.131258:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.131299:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660212224:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845024.131392:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000004:00010000:0.0:1788845024.148120:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845023 -> 1788845024 00000004:00010000:0.0:1788845024.148132:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845023 -> 1788845024 00000100:00100000:0.0:1788845024.148160:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660212224:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845024.148187:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:0.0:1788845024.148208:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ac589 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ee0 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.148231:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.148234:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845024.148240:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ac589 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ee0 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.148256:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:0.0:1788845024.148329:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641468800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17150us (17385us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845024.148338:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 793 00000100:00100000:2.0:1788845024.157947:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.158101:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641468928 00000100:00100000:0.0:1788845024.158112:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641468928 00000100:00100000:0.0:1788845024.158174:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 794 00000100:00100000:0.0:1788845024.158185:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641468928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.158207:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.158229:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ac590 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ee7 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.158273:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.158315:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660212352:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845024.158406:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845024.164399:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660212352:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845024.164428:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:0.0:1788845024.164447:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ac590 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ee7 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.164467:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.164470:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845024.164475:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ac590 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ee7 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.164489:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:0.0:1788845024.164566:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641468928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6383us (6630us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845024.164575:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 794 00000100:00100000:1.0:1788845024.169260:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.169347:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641469056 00000100:00100000:2.0:1788845024.169359:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641469056 00000100:00100000:2.0:1788845024.169429:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 514 00000100:00100000:2.0:1788845024.169439:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641469056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.169460:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.169480:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ac597 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7eee expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.169580:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660212480:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.169664:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1d80 going to sleep for 11 seconds 00002000:00010000:1.0:1788845024.177657:0:6685:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1984146 00002000:00010000:1.0:1788845024.177665:0:6685:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845023 -> 1788845024 00002000:00010000:1.0:1788845024.177668:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845023 -> 1788845024 00002000:00010000:1.0:1788845024.177670:0:6685:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845023 -> 1788845024 00000100:00100000:1.0:1788845024.177714:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660212480:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845024.177757:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ac597 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eee expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.177782:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ac597 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7eee expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.177797:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:1.0:1788845024.178067:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641469056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8436us (8623us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845024.178077:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 514 00000100:00100000:1.0:1788845024.180815:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.180922:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641469184 00000100:00100000:0.0:1788845024.180932:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641469184 00000100:00100000:0.0:1788845024.181010:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 515 00000100:00100000:0.0:1788845024.181021:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641469184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.181042:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.181063:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ac59e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ef5 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.181124:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660212608:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845024.181212:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845024.185369:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660212608:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845024.185417:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ac59e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ef5 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.185441:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ac59e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ef5 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.185455:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:0.0:1788845024.185544:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641469184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4525us (4740us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845024.185554:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 515 00000100:00100000:3.0:1788845024.193699:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.193915:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641469312 00000100:00100000:2.0:1788845024.193927:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641469312 00002000:00100000:2.0:1788845024.193980:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1941504->1984145) req@ffff949f03411180 x1875739641469312/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.193991:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03411180 with x1875739641469312 ext(1941504->1984145) 00002000:00010000:2.0:1788845024.194008:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03411180 00000100:00100000:2.0:1788845024.194035:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 304 00000100:00100000:2.0:1788845024.194045:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641469312:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845024.199740:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641469312:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5686us (6042us total) trans 4294967396 rc 0/0 00000100:00100000:2.0:1788845024.199754:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 304 00002000:00100000:2.0:1788845024.199767:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1941504->1984145) req@ffff949f03411180 x1875739641469312/t4294967396(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:130/0 lens 488/448 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.199778:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03411180 with x1875739641469312 ext(1941504->1984145) 00002000:00010000:2.0:1788845024.199784:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03411180 00000100:00100000:1.0:1788845024.203189:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.205366:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641469440 00000100:00100000:3.0:1788845024.205381:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641469440 00000100:00100000:2.0:1788845024.205800:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 305 00000100:00100000:2.0:1788845024.205812:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641469440:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:2.0:1788845024.206136:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1984146 -> 1984146 00002000:00010000:2.0:1788845024.206140:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845024 -> 1788845024 00002000:00010000:2.0:1788845024.206142:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845024 -> 1788845023 00002000:00010000:2.0:1788845024.206156:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845024 -> 1788845024 00002000:00010000:2.0:1788845024.206158:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 137 -> 137 00000100:00100000:2.0:1788845024.206219:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641469440:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 409us (3039us total) trans 4294967397 rc 0/0 00000100:00100000:2.0:1788845024.206227:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 305 00000100:00100000:3.0:1788845024.216346:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.216439:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641469568 00000100:00100000:2.0:1788845024.216450:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641469568 00000100:00100000:2.0:1788845024.216511:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 516 00000100:00100000:2.0:1788845024.216523:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641469568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.216551:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.216571:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac5a5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7efc expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.216633:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660212736:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.216726:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1600 going to sleep for 11 seconds 00002000:00010000:2.0:1788845024.223761:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845023 -> 1788845024 00000100:00100000:2.0:1788845024.223795:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660212736:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845024.223834:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac5a5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7efc expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.223861:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac5a5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7efc expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.223878:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845024.224093:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641469568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7570us (7754us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.224127:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 516 00000100:00100000:0.0:1788845024.233986:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.234091:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641469696 00000100:00100000:2.0:1788845024.234104:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641469696 00000100:00100000:2.0:1788845024.234169:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 517 00000100:00100000:2.0:1788845024.234181:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641469696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.234200:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.234219:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac5ac lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f03 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.234280:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660212864:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.234363:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.237502:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660212864:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845024.237552:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac5ac lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f03 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.237577:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ac5ac lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f03 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.237591:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845024.237647:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641469696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3468us (3671us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.237655:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 517 00000100:00100000:3.0:1788845024.256170:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845024.256268:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641469824 00000100:00100000:1.0:1788845024.256277:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641469824 00000100:00100000:1.0:1788845024.256332:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 518 00000100:00100000:1.0:1788845024.256342:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641469824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845024.256363:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845024.256381:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d8a87e9c/0xb2187301287ac5b3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f0a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845024.256437:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660212992:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845024.256512:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845024.263005:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660212992:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845024.263056:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d8a87e9c/0xb2187301287ac5b3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f0a expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.263105:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8a87e9c/0xb2187301287ac5b3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f0a expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.263120:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2600, rc 0) 00000100:00100000:1.0:1788845024.263194:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641469824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6852us (7031us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845024.263203:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 518 00000100:00100000:3.0:1788845024.266440:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.266551:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641469952 00000100:00100000:0.0:1788845024.266561:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641469952 00000100:00100000:0.0:1788845024.266623:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 519 00000100:00100000:0.0:1788845024.266632:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641469952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.266661:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.266678:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ac5ba lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f11 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.266732:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660213120:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845024.266804:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845024.273755:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660213120:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845024.273796:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ac5ba lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f11 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.273835:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ac5ba lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f11 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.273850:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:0.0:1788845024.273964:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641469952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7332us (7533us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845024.273973:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 519 00000100:00100000:1.0:1788845024.297176:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.297281:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641470080 00000100:00100000:0.0:1788845024.297291:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641470080 00002000:00100000:0.0:1788845024.297343:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d0e00 x1875739641470080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845024.297353:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d0e00 with x1875739641470080 00000100:00100000:0.0:1788845024.297394:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 476 00000100:00100000:0.0:1788845024.297404:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641470080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845024.327239:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641470080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 29823us (30058us total) trans 0 rc 995328/995328 00000100:00100000:2.0:1788845024.327275:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 476 00002000:00100000:2.0:1788845024.327294:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d0e00 x1875739641470080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/440 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.327306:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d0e00 with x1875739641470080 00000100:00100000:3.0:1788845024.333665:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.333834:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641470208 00000100:00100000:2.0:1788845024.333846:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641470208 00000100:00100000:2.0:1788845024.333924:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 520 00000100:00100000:2.0:1788845024.333935:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641470208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.333953:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.333971:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000591c18bd/0xb2187301287ac5c1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f18 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.334051:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660213248:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.334141:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.341254:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660213248:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845024.341298:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000591c18bd/0xb2187301287ac5c1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f18 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.341323:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000591c18bd/0xb2187301287ac5c1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f18 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.341338:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0400, rc 0) 00000100:00100000:2.0:1788845024.341404:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641470208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7470us (7747us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.341433:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 520 00000100:00100000:0.0:1788845024.350978:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.351105:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641470336 00000100:00100000:2.0:1788845024.351115:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641470336 00000100:00100000:2.0:1788845024.351172:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 521 00000100:00100000:2.0:1788845024.351182:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641470336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.351201:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.351219:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000591c18bd/0xb2187301287ac5c8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f1f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.351278:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660213376:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.351372:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.353731:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660213376:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845024.353768:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000591c18bd/0xb2187301287ac5c8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f1f expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.353788:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000591c18bd/0xb2187301287ac5c8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f1f expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.353803:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0400, rc 0) 00000100:00100000:2.0:1788845024.354069:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641470336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2887us (3103us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.354076:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 521 00000100:00100000:1.0:1788845024.363484:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.363584:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641470464 00000100:00100000:3.0:1788845024.363594:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641470464 00000100:00100000:3.0:1788845024.363648:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 795 00000100:00100000:3.0:1788845024.363659:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2367:x1875739641470464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845024.363681:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845024.363702:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5cf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f26 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.363741:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845024.363793:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03412d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660213504:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845024.363900:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000004:00010000:3.0:1788845024.368851:0:5840:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845023 -> 1788845024 00000100:00100000:3.0:1788845024.368929:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03412d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660213504:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845024.368955:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:3.0:1788845024.368975:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5cf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f26 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.369008:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845024.369012:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845024.369017:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5cf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f26 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.369031:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:3.0:1788845024.369108:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2367:x1875739641470464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5449us (5635us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845024.369117:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 795 00000100:00100000:1.0:1788845024.377285:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.377375:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641470592 00000100:00100000:3.0:1788845024.377384:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641470592 00000100:00100000:3.0:1788845024.377448:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 796 00000100:00100000:3.0:1788845024.377458:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641470592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845024.377480:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845024.377502:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5d6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f2d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.377543:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845024.377588:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03412d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660213632:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845024.377676:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845024.383472:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03412d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660213632:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845024.383501:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:3.0:1788845024.383524:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5d6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f2d expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.383544:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845024.383547:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845024.383552:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5d6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f2d expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.383568:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:3.0:1788845024.383648:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641470592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6190us (6372us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845024.383657:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 796 00000100:00100000:1.0:1788845024.390242:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.390354:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641470848 00000100:00100000:0.0:1788845024.390367:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641470848 00000100:00100000:0.0:1788845024.390461:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 797 00000100:00100000:0.0:1788845024.390472:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641470848:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845024.390529:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845024.390557:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ac543 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ea1 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.390578:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.390582:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845024.390785:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.390962:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.390973:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.391018:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.391038:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 19 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.391053:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660213760:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845024.391204:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fe40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845024.399500:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.403336:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660213760:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845024.403377:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.403933:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641470976 00000100:00100000:2.0:1788845024.403942:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641470976 00010000:00010000:2.0:1788845024.404011:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 20 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.404054:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 289 00000100:00100000:2.0:1788845024.404064:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641470976:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845024.404080:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845024.404087:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 20 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.404105:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 20 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.404128:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac56d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ec4 expref: 20 pid: 5840 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.404143:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845024.404148:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.404158:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.404165:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.404185:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845024.404256:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641470976:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 192us (4767us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845024.404264:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 289 00010000:00010000:0.0:1788845024.404981:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.405001:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845024.405126:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967620 00000004:00010000:0.0:1788845024.405180:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991973 -> 991973 00000004:00010000:0.0:1788845024.405184:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845024 -> 1788845024 00000004:00010000:0.0:1788845024.405186:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845024 -> 1788845023 00000004:00010000:0.0:1788845024.405189:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845024 -> 1788845024 00000004:00010000:0.0:1788845024.405191:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:0.0:1788845024.405199:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004baa0d46 reply state 00000000183fe964 transno 4294967621 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.405209:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845024.405308:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641470848:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 14836us (15073us total) trans 4294967621 rc 0/0 00000100:00100000:0.0:1788845024.405316:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 797 00000020:00080000:0.0:1788845024.405321:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788845024.411611:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845024.412095:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.412109:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.412117:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.412123:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191f200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845024.412128:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.412136:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.412156:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845024.412160:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845024.412164:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.412170:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ac5dd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.412244:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641471104 00000100:00100000:0.0:1788845024.412255:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641471104 00000100:00100000:0.0:1788845024.412355:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 798 00000100:00100000:0.0:1788845024.412368:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:14891:x1875739641471104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.412408:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.412426:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f34 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.412455:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.412465:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.412498:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.412528:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 18 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.412543:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660213888:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845024.412645:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845024.415214:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660213888:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845024.415259:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.415277:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.415285:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845024.415290:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.415322:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:0.0:1788845024.415432:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641471104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3066us (3833us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845024.415638:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 798 00000100:00100000:1.0:1788845024.419717:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.428316:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641471232 00000100:00100000:0.0:1788845024.428332:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641471232 00002000:00100000:0.0:1788845024.428396:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848380 x1875739641471232/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845024.428407:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848380 with x1875739641471232 00010000:00010000:0.0:1788845024.428418:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.428438:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 19 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845024.428446:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f03848380 00000100:00100000:0.0:1788845024.428483:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 18 00000100:00100000:0.0:1788845024.428492:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641471232:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845024.433844:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991973 -> 991973 00000004:00010000:0.0:1788845024.433854:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845024 -> 1788845024 00000004:00010000:0.0:1788845024.433857:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:0.0:1788845024.433859:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845024 -> 1788845024 00000004:00010000:0.0:1788845024.433861:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845024.433984:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641471232:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5487us (14273us total) trans 4294967622 rc 0/0 00000100:00100000:0.0:1788845024.434013:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 18 00002000:00100000:0.0:1788845024.434028:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848380 x1875739641471232/t4294967622(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/448 e 0 to 0 dl 1788845035 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845024.434038:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848380 with x1875739641471232 00010000:00010000:0.0:1788845024.434049:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 21 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.434069:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 21 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845024.434077:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f03848380 00000100:00100000:3.0:1788845024.449298:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845024.449413:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641471360 00000100:00100000:1.0:1788845024.449424:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641471360 00010000:00010000:1.0:1788845024.449469:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 21 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845024.449516:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 290 00000100:00100000:1.0:1788845024.449528:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641471360:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845024.449543:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845024.449572:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 21 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.449612:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 21 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.449636:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ac582 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ed9 expref: 21 pid: 5839 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.449654:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845024.449660:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.449671:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.449680:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.449717:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845024.449731:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660214016:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845024.449975:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.463428:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660214016:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845024.463456:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845024.463513:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641471360:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 13986us (14224us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845024.463526:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 290 00000100:00100000:0.0:1788845024.464138:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.466101:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641471488 00000100:00100000:2.0:1788845024.466117:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641471488 00002000:00010000:2.0:1788845024.466167:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1993459->18446744073709551615) 00002000:00100000:2.0:1788845024.466172:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03532b480 with x1875739641471488 ext(1993459->18446744073709551615) 00002000:00010000:2.0:1788845024.466186:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03532b480. 00000100:00100000:2.0:1788845024.466217:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 306 00000100:00100000:2.0:1788845024.466227:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641471488:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845024.466492:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1984146 -> 1993459 00002000:00010000:2.0:1788845024.466496:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845024 -> 1788845024 00002000:00010000:2.0:1788845024.466499:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845024 -> 1788845023 00002000:00010000:2.0:1788845024.466501:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845024 -> 1788845024 00002000:00010000:2.0:1788845024.466503:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 137 -> 137 00000100:00100000:2.0:1788845024.466725:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641471488:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 497us (2586us total) trans 4294967398 rc 0/0 00000100:00100000:2.0:1788845024.466734:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 306 00002000:00010000:2.0:1788845024.466738:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1993459->18446744073709551615) 00002000:00100000:2.0:1788845024.466741:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03532b480 with x1875739641471488 ext(1993459->18446744073709551615) 00002000:00010000:2.0:1788845024.466746:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03532b480. 00000100:00100000:3.0:1788845024.484731:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.484844:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641471616 00000100:00100000:2.0:1788845024.484854:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641471616 00002000:00100000:2.0:1788845024.484954:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410e00 x1875739641471616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.484981:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410e00 with x1875739641471616 00000100:00100000:2.0:1788845024.485055:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 477 00000100:00100000:2.0:1788845024.485066:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641471616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845024.522207:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.522379:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641471744 00000100:00100000:0.0:1788845024.522390:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641471744 00002000:00100000:0.0:1788845024.522444:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1994751) req@ffff949f02360e00 x1875739641471744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:130/0 lens 504/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845024.522455:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360e00 with x1875739641471744 ext(1048576->1994751) 00002000:00010000:0.0:1788845024.522460:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360e00 00000100:00100000:0.0:1788845024.522494:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 307 00000100:00100000:0.0:1788845024.522503:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641471744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845024.527113:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641471616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 42033us (42377us total) trans 0 rc 995328/995328 00000100:00100000:1.0:1788845024.527132:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 477 00002000:00100000:1.0:1788845024.527147:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410e00 x1875739641471616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:130/0 lens 488/440 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845024.527165:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410e00 with x1875739641471616 00000100:00100000:0.0:1788845024.552116:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641471744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 29602us (29908us total) trans 0 rc 917504/917504 00000100:00100000:0.0:1788845024.552130:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 307 00002000:00100000:0.0:1788845024.552146:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1994751) req@ffff949f02360e00 x1875739641471744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:130/0 lens 504/440 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 917504/917504 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845024.552174:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360e00 with x1875739641471744 ext(1048576->1994751) 00002000:00010000:0.0:1788845024.552180:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360e00 00000100:00100000:1.0:1788845024.576346:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.576541:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641471872 00000100:00100000:2.0:1788845024.576550:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641471872 00002000:00100000:2.0:1788845024.576592:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1933312->1993458) req@ffff949f03849c00 x1875739641471872/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.576600:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03849c00 with x1875739641471872 ext(1933312->1993458) 00002000:00010000:2.0:1788845024.576605:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03849c00 00000100:00100000:2.0:1788845024.576634:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 308 00000100:00100000:2.0:1788845024.576640:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641471872:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845024.587182:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641471872:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 10529us (10832us total) trans 4294967399 rc 0/0 00000100:00100000:2.0:1788845024.587196:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 308 00002000:00100000:2.0:1788845024.587209:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1933312->1993458) req@ffff949f03849c00 x1875739641471872/t4294967399(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:130/0 lens 488/448 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.587220:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03849c00 with x1875739641471872 ext(1933312->1993458) 00002000:00010000:2.0:1788845024.587226:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03849c00 00000100:00100000:1.0:1788845024.591656:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.591867:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641472000 00000100:00100000:2.0:1788845024.591877:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641472000 00000100:00100000:2.0:1788845024.591959:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 522 00000100:00100000:2.0:1788845024.591970:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641472000:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:0.0:1788845024.712032:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967399 is committed 00000001:00080000:0.0:1788845024.712070:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967393 is committed 00000001:00080000:0.0:1788845024.712078:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967394 is committed 00000100:00100000:2.0:1788845024.712170:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641472000:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 120196us (120516us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845024.712183:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 522 00000100:00100000:0.0:1788845024.720581:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.720691:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641472128 00000100:00100000:3.0:1788845024.720701:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641472128 00000100:00100000:3.0:1788845024.720982:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 799 00000100:00100000:3.0:1788845024.721018:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641472128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845024.721039:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845024.721061:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac5eb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f3b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.721154:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5f2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.721170:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.721180:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845024.721206:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1993459 00000004:00200000:3.0:1788845024.721210:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845024.721228:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.721256:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ac5eb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f3b expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.721274:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5f2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f3b expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.721283:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845024.721286:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845024.721289:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:3.0:1788845024.721359:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:14891:x1875739641472128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 344us (790us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845024.721375:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 799 00000100:00100000:0.0:1788845024.728621:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845024.728870:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641472256 00000100:00100000:1.0:1788845024.728922:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641472256 00000100:00100000:0.0:1788845024.736167:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 800 00000100:00100000:0.0:1788845024.736184:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641472256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.736207:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.736225:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f42 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.736251:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.736260:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.736287:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.736301:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 20 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.736311:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660214144:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845024.736423:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00000100:00100000:1.0:1788845024.741551:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.741728:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641472384 00000100:00100000:2.0:1788845024.741738:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641472384 00010000:00010000:2.0:1788845024.741787:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 21 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.741834:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 291 00000100:00100000:2.0:1788845024.741845:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641472384:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845024.741859:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845024.741924:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 21 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.741945:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 21 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.741967:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845024.741974:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.742002:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.742014:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.742036:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ac5e4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f34 expref: 21 pid: 5841 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.742065:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660214144:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845024.742091:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.742100:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.742112:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f656300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660214272:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845024.742115:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.742120:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845024.742122:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960000, rc 0) 00000100:00100000:0.0:1788845024.742180:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641472256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6003us (13570us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845024.742189:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 800 00000100:00100000:2.0:1788845024.742230:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845024.762203:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.762273:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641472512 00000100:00100000:3.0:1788845024.762282:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641472512 00000100:00100000:3.0:1788845024.762415:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:2.0:1788845024.763878:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f656300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660214272:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845024.763944:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845024.764014:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641472384:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 22170us (22472us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845024.764025:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 291 00000100:00100000:3.0:1788845024.764083:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641472512:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845024.764363:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991973 -> 991973 00000004:00010000:3.0:1788845024.764368:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845024 -> 1788845024 00000004:00010000:3.0:1788845024.764371:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:3.0:1788845024.764373:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845024 -> 1788845024 00000004:00010000:3.0:1788845024.764376:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845024.764432:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:2369:x1875739641472512:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 2007us (2238us total) trans 4294967623 rc 0/0 00000100:00100000:3.0:1788845024.764440:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:1.0:1788845024.777795:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.777913:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641472640 00000100:00100000:3.0:1788845024.777927:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641472640 00000100:00100000:0.0:1788845024.778026:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 801 00000100:00100000:0.0:1788845024.778041:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641472640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.778072:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.778095:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960400/0xb2187301287ac600 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f49 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.778190:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.778206:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.778219:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845024.778264:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1993459 00000004:00200000:0.0:1788845024.778269:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845024.778291:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.778320:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960400/0xb2187301287ac600 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.778340:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.778348:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.778351:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845024.778355:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960200, rc 0) 00000100:00100000:0.0:1788845024.778436:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641472640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 400us (651us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845024.778443:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 801 00000100:00100000:3.0:1788845024.782834:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.782976:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641472768 00000100:00100000:0.0:1788845024.782995:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641472768 00000100:00100000:0.0:1788845024.783050:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 523 00000100:00100000:0.0:1788845024.783061:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641472768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.783081:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.783099:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287ac60e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f50 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.783158:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660214400:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845024.783264:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fe40 going to sleep for 11 seconds 00002000:00010000:0.0:1788845024.786637:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845023 -> 1788845024 00000100:00100000:0.0:1788845024.786674:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660214400:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845024.786717:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287ac60e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f50 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.786743:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287ac60e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f50 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.786756:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961000, rc 0) 00000100:00100000:0.0:1788845024.787122:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641472768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4060us (4295us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845024.787132:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 523 00000100:00100000:2.0:1788845024.796252:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.796340:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641472896 00000100:00100000:3.0:1788845024.796350:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641472896 00000100:00100000:3.0:1788845024.796406:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 524 00000100:00100000:3.0:1788845024.796416:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641472896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845024.796435:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845024.796455:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac615 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f57 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845024.796516:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660214528:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845024.796624:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845024.801266:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660214528:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845024.801312:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac615 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f57 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.801340:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ac615 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f57 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.801353:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:3.0:1788845024.801421:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641472896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5006us (5178us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845024.801430:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 524 00000100:00100000:0.0:1788845024.810311:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845024.811867:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641473024 00000100:00100000:1.0:1788845024.811896:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641473024 00002000:00100000:1.0:1788845024.811958:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035328a80 x1875739641473024/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845024.811970:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328a80 with x1875739641473024 00000100:00100000:1.0:1788845024.812025:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 479 00000100:00100000:1.0:1788845024.812036:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641473024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845024.812288:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641473024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (1986us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845024.812295:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 479 00002000:00100000:1.0:1788845024.812302:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035328a80 x1875739641473024/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/4536 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845024.812309:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328a80 with x1875739641473024 00000100:00100000:2.0:1788845024.821868:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.822052:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641473152 00000100:00100000:0.0:1788845024.822062:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641473152 00002000:00100000:0.0:1788845024.822189:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741f80 x1875739641473152/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845024.822201:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741f80 with x1875739641473152 00000100:00100000:0.0:1788845024.822309:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 480 00000100:00100000:0.0:1788845024.822321:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641473152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845024.822598:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641473152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (737us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845024.822605:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 480 00002000:00100000:0.0:1788845024.822613:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741f80 x1875739641473152/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/4536 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845024.822635:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741f80 with x1875739641473152 00000100:00100000:3.0:1788845024.831855:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845024.833863:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641473280 00000100:00100000:1.0:1788845024.833878:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641473280 00000100:00100000:1.0:1788845024.833981:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 802 00000100:00100000:1.0:1788845024.834001:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641473280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845024.834039:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845024.834063:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac61c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f5e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.834101:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845024.834146:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660214656:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845024.834233:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845024.841248:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845023 -> 1788845024 00000100:00100000:2.0:1788845024.841279:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660214656:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845024.841303:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:2.0:1788845024.841321:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac61c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f5e expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.841341:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845024.841344:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845024.841348:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac61c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f5e expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.841363:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2800, rc 0) 00000100:00100000:2.0:1788845024.841493:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641473280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7492us (9648us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.841514:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 802 00000100:00100000:2.0:1788845024.846324:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.846414:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641473408 00000100:00100000:0.0:1788845024.846424:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641473408 00000100:00100000:0.0:1788845024.846499:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 803 00000100:00100000:0.0:1788845024.846520:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641473408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.846545:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.846568:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961200/0xb2187301287ac623 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f65 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.846609:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.846654:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660214784:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845024.846764:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00010000:00010000:1.0:1788845024.850676:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845024.866773:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660214784:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845024.866806:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:0.0:1788845024.866827:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961200/0xb2187301287ac623 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f65 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.866848:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.866852:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845024.866856:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961200/0xb2187301287ac623 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f65 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.866915:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961200, rc 0) 00000100:00100000:0.0:1788845024.866982:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641473408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 20473us (20668us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845024.867018:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 803 00000100:00100000:1.0:1788845024.875289:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.875393:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641473536 00000100:00100000:2.0:1788845024.875404:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641473536 00000100:00100000:2.0:1788845024.875466:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 525 00000100:00100000:2.0:1788845024.875476:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641473536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.875495:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.875523:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac62a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f6c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.875582:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660214912:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.875677:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845024.879038:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660214912:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845024.879084:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac62a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f6c expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.879109:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac62a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f6c expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.879123:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0a00, rc 0) 00000100:00100000:0.0:1788845024.879188:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641473536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3712us (3910us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845024.879197:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 525 00000100:00100000:1.0:1788845024.887161:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.887268:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641473664 00000100:00100000:2.0:1788845024.887278:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641473664 00000100:00100000:2.0:1788845024.887338:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 526 00000100:00100000:2.0:1788845024.887348:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641473664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.887368:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.887386:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac631 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f73 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.887444:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660215040:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.887539:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.891087:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660215040:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845024.891130:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac631 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f73 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.891174:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac631 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f73 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.891187:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:2.0:1788845024.891248:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641473664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3900us (4096us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.891268:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 526 00000100:00100000:1.0:1788845024.896738:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.896952:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641473792 00000100:00100000:2.0:1788845024.896963:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641473792 00002000:00100000:2.0:1788845024.897024:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848700 x1875739641473792/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.897037:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848700 with x1875739641473792 00000100:00100000:2.0:1788845024.897076:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 481 00000100:00100000:2.0:1788845024.897087:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641473792:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845024.897327:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641473792:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 241us (607us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845024.897335:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 481 00002000:00100000:2.0:1788845024.897342:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848700 x1875739641473792/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/4536 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845024.897349:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848700 with x1875739641473792 00000100:00100000:1.0:1788845024.903013:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.903103:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641473920 00000100:00100000:3.0:1788845024.903114:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641473920 00002000:00100000:3.0:1788845024.903162:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384ad80 x1875739641473920/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845024.903174:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384ad80 with x1875739641473920 00000100:00100000:3.0:1788845024.903201:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 482 00000100:00100000:3.0:1788845024.903211:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641473920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845024.903518:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641473920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 306us (515us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845024.903526:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 482 00002000:00100000:3.0:1788845024.903533:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384ad80 x1875739641473920/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 488/4536 e 0 to 0 dl 1788845035 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845024.903555:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384ad80 with x1875739641473920 00000100:00100000:1.0:1788845024.914494:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.914617:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641474048 00000100:00100000:0.0:1788845024.914628:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641474048 00000100:00100000:0.0:1788845024.914708:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 804 00000100:00100000:0.0:1788845024.914719:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641474048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845024.914752:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845024.914774:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961200/0xb2187301287ac638 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f7a expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.914811:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845024.914853:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660215168:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845024.915009:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845024.926348:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660215168:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845024.926380:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:0.0:1788845024.926408:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961200/0xb2187301287ac638 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f7a expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.926431:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845024.926434:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845024.926439:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961200/0xb2187301287ac638 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f7a expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845024.926455:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961200, rc 0) 00000100:00100000:0.0:1788845024.926519:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641474048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11801us (12036us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845024.926528:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 804 00000100:00100000:1.0:1788845024.937117:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.937214:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641474176 00000100:00100000:2.0:1788845024.937225:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641474176 00000100:00100000:2.0:1788845024.937284:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 805 00000100:00100000:2.0:1788845024.937310:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2368:x1875739641474176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.937332:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.937351:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac63f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f81 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.937388:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.937429:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660215296:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845024.937515:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.941422:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660215296:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845024.941450:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991973 00010000:00010000:2.0:1788845024.941468:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac63f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f81 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.941488:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845024.941491:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845024.941496:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac63f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f81 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.941521:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:2.0:1788845024.941585:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2368:x1875739641474176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4276us (4476us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845024.941593:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 805 00000100:00100000:1.0:1788845024.945309:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.945398:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641474304 00000100:00100000:2.0:1788845024.945409:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641474304 00000100:00100000:2.0:1788845024.945470:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 806 00000100:00100000:2.0:1788845024.945491:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641474304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845024.945525:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845024.945546:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f88 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.945576:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.945583:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.945609:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.945621:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 22 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.945630:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660215424:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845024.945702:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845024.948805:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660215424:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845024.948840:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845024.948861:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:2.0:1788845024.948870:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845024.948873:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845024.948878:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845024.948978:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641474432 00010000:00010000:2.0:1788845024.948979:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:2.0:1788845024.949045:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:14891:x1875739641474304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3556us (3744us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845024.949054:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 806 00000100:00100000:3.0:1788845024.950123:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641474432 00002000:00100000:3.0:1788845024.950171:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384aa00 x1875739641474432/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 504/0 e 0 to 0 dl 1788845035 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845024.950181:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384aa00 with x1875739641474432 00010000:00010000:3.0:1788845024.950189:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 23 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.950209:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 23 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845024.950218:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0384aa00 00000100:00100000:3.0:1788845024.950303:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 19 00000100:00100000:3.0:1788845024.950314:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641474432:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845024.957249:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991973 -> 991973 00000004:00010000:3.0:1788845024.957259:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845024 -> 1788845024 00000004:00010000:3.0:1788845024.957261:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845024 -> 1788845023 00000004:00010000:3.0:1788845024.957263:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845024 -> 1788845024 00000004:00010000:3.0:1788845024.957265:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845024.957355:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641474432:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 7037us (8488us total) trans 4294967624 rc 0/0 00000100:00100000:3.0:1788845024.957367:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 19 00002000:00100000:3.0:1788845024.957380:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384aa00 x1875739641474432/t4294967624(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:130/0 lens 504/448 e 0 to 0 dl 1788845035 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845024.957403:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384aa00 with x1875739641474432 00010000:00010000:3.0:1788845024.957415:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 25 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845024.957435:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 25 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845024.957442:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0384aa00 00000100:00100000:0.0:1788845024.968837:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845024.969028:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641474560 00000100:00100000:2.0:1788845024.969045:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641474560 00010000:00010000:2.0:1788845024.969107:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 25 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.969159:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 292 00000100:00100000:2.0:1788845024.969168:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008915050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641474560:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845024.969181:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845024.969362:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 25 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.969381:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 25 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.969404:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960000/0xb2187301287ac5f9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f42 expref: 25 pid: 5839 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.969421:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845024.969427:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.969460:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.969467:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845024.969505:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845024.969518:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660215552:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845024.969610:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845024.972941:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660215552:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845024.972969:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845024.973152:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008915050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641474560:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3983us (4330us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845024.973165:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 292 00000100:00100000:1.0:1788845024.975440:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845024.976172:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641474688 00000100:00100000:3.0:1788845024.976191:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641474688 00000100:00100000:3.0:1788845024.976296:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 483 00000100:00100000:3.0:1788845024.976308:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641474688:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845024.976755:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991973 -> 991973 00000004:00010000:3.0:1788845024.976760:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845024 -> 1788845024 00000004:00010000:3.0:1788845024.976763:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:3.0:1788845024.976766:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845024 -> 1788845024 00000004:00010000:3.0:1788845024.976768:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845024.977287:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641474688:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 976us (1851us total) trans 4294967625 rc 0/0 00000100:00100000:3.0:1788845024.977300:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 483 00000100:00100000:1.0:1788845024.989326:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845024.989950:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641474816 00000100:00100000:0.0:1788845024.989965:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641474816 00000100:00100000:0.0:1788845024.991717:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 807 00000100:00100000:1.0:1788845024.992555:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641474816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845024.992597:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845024.992632:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f8f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.992687:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.992721:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.992767:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845024.992792:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 21 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845024.992810:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660215680:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845024.993210:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.003831:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660215680:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845025.003908:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.003931:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.003950:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845025.003956:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.003968:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3e00, rc 0) 00000100:00100000:3.0:1788845025.004052:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641474816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12323us (14741us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.004060:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 807 00000100:00100000:2.0:1788845025.013387:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.013576:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641474944 00000100:00100000:1.0:1788845025.013594:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641474944 00010000:00010000:1.0:1788845025.013667:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 22 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.013714:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 293 00000100:00100000:1.0:1788845025.013727:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641474944:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845025.013742:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845025.013770:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 22 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1788845025.013912:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788845025.014115:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 22 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.014142:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac646 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f88 expref: 22 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.014177:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845025.014185:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.014197:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.014208:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.014251:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.015299:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660215808:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845025.015408:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.023469:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660215808:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845025.023497:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845025.023585:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641474944:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9861us (10213us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.023597:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 293 00000100:00100000:3.0:1788845025.034550:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.034808:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641475072 00000100:00100000:1.0:1788845025.034818:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641475072 00000100:00080000:0.0:1788845025.034915:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00100000:1.0:1788845025.034948:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03412a00 x1875739641475072/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845025.034960:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03412a00 with x1875739641475072 00000100:00100000:1.0:1788845025.035009:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 484 00000100:00100000:1.0:1788845025.035021:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2370:x1875739641475072:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845025.053221:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2370:x1875739641475072:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 18193us (18670us total) trans 0 rc 995328/995328 00000100:00100000:2.0:1788845025.053235:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 484 00002000:00100000:2.0:1788845025.053248:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03412a00 x1875739641475072/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:131/0 lens 488/440 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845025.053258:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03412a00 with x1875739641475072 00000100:00100000:3.0:1788845025.062762:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.063044:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641475200 00000100:00100000:0.0:1788845025.063072:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641475200 00000100:00100000:0.0:1788845025.063149:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 527 00000100:00100000:0.0:1788845025.063161:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641475200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.063181:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.063198:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac654 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f96 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.063260:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660215936:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.063529:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00080000:1.0:1788845025.063595:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845025.090328:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660215936:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845025.090375:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac654 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f96 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.090401:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac654 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f96 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.090440:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3200, rc 0) 00000100:00100000:2.0:1788845025.090652:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641475200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 27493us (27898us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.090671:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 527 00000100:00100000:0.0:1788845025.095289:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.095376:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641475328 00000100:00100000:3.0:1788845025.095387:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641475328 00000100:00100000:3.0:1788845025.095451:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 528 00000100:00100000:3.0:1788845025.095461:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641475328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.095478:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.095497:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac65b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7f9d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.095570:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03413100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660216064:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845025.096258:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845025.100234:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03413100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660216064:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845025.100274:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac65b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f9d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.100298:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ac65b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f9d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.100312:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:0.0:1788845025.100384:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641475328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4924us (5104us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.100394:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 528 00000100:00100000:0.0:1788845025.107913:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.108083:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641475584 00000100:00100000:3.0:1788845025.108105:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641475584 00000100:00100000:3.0:1788845025.108167:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 808 00000100:00100000:3.0:1788845025.108177:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:14891:x1875739641475584:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845025.108229:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845025.108255:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac5f2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f3b expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.108273:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.108277:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845025.108480:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.108494:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.108503:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.108535:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.108552:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 24 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.108565:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660216192:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845025.108652:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:1.0:1788845025.137430:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.137682:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660216192:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845025.137708:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.139136:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641475712 00000100:00100000:2.0:1788845025.139149:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641475712 00010000:00010000:2.0:1788845025.139194:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 25 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.139239:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 294 00000100:00100000:2.0:1788845025.139249:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641475712:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845025.139273:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845025.139280:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 25 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.139298:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 25 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.139319:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac607 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f49 expref: 25 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.139336:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.139341:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.139366:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.139373:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.139403:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845025.139469:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641475712:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 221us (2050us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.139477:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 294 00010000:00010000:3.0:1788845025.140228:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.140239:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845025.140368:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967626 00000004:00010000:3.0:1788845025.140417:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991973 -> 991973 00000004:00010000:3.0:1788845025.140420:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845024 -> 1788845025 00000004:00010000:3.0:1788845025.140423:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:3.0:1788845025.140426:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845024 -> 1788845025 00000004:00010000:3.0:1788845025.140428:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:3.0:1788845025.140435:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a7c62e65 reply state 0000000080cdce76 transno 4294967627 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.140444:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845025.140546:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641475584:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 32368us (32670us total) trans 4294967627 rc 0/0 00000100:00100000:3.0:1788845025.140553:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 808 00000020:00080000:3.0:1788845025.140557:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845025.150438:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.150458:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.150466:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.150490:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845025.150504:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.150513:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.150530:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.150533:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845025.150537:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.150544:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac662 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.153728:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.153932:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641475840 00000100:00100000:2.0:1788845025.153945:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641475840 00000100:00100000:2.0:1788845025.154014:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 809 00000100:00100000:2.0:1788845025.154024:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:14891:x1875739641475840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.154051:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.154072:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fa4 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.154105:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.154117:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.154154:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.154173:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 23 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.154197:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660216320:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845025.154287:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845025.165185:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.165277:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641475968 00000100:00100000:3.0:1788845025.165286:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641475968 00010000:00010000:3.0:1788845025.165326:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 24 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.165371:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 295 00000100:00100000:3.0:1788845025.165381:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2370:x1875739641475968:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845025.165397:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845025.165427:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 24 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.165445:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 24 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.165488:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.165495:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.165517:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.165526:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.165562:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.165577:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660216448:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845025.165756:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00010000:00010000:2.0:1788845025.168133:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3e00/0xb2187301287ac64d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7f8f expref: 24 pid: 11036 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.168160:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660216320:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845025.168194:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.168210:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.168214:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845025.168217:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845025.168279:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641475840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 14257us (14563us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.168288:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 809 00000100:00100000:1.0:1788845025.177375:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.177443:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641476096 00000100:00100000:2.0:1788845025.177454:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641476096 00002000:00010000:2.0:1788845025.177496:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845025.177512:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98380 with x1875739641476096 00000100:00100000:2.0:1788845025.177549:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 485 00000100:00100000:2.0:1788845025.177559:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641476096:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:2.0:1788845025.177751:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991973 -> 1002492 00000004:00010000:2.0:1788845025.177755:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845025 -> 1788845025 00000004:00010000:2.0:1788845025.177758:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845025.177770:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845025 -> 1788845025 00000004:00010000:2.0:1788845025.177773:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845025.177854:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2369:x1875739641476096:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 296us (487us total) trans 4294967628 rc 0/0 00000100:00100000:2.0:1788845025.177862:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 485 00002000:00010000:2.0:1788845025.177866:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845025.177868:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98380 with x1875739641476096 00000100:00100000:3.0:1788845025.178999:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660216448:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845025.179024:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845025.179077:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641475968:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 13697us (13901us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.179087:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 295 00000100:00100000:1.0:1788845025.195453:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.195611:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641476224 00000100:00100000:0.0:1788845025.195622:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641476224 00002000:00010000:0.0:1788845025.195683:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845025.195687:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a4b100 with x1875739641476224 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845025.195711:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a4b100. 00000100:00100000:0.0:1788845025.195748:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 309 00000100:00100000:0.0:1788845025.195759:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02a4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641476224:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845025.196837:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1993459 -> 0 00002000:00010000:0.0:1788845025.196844:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845024 -> 1788845025 00002000:00010000:0.0:1788845025.196847:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845024 -> 1788845023 00002000:00010000:0.0:1788845025.196849:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845024 -> 1788845025 00002000:00010000:0.0:1788845025.196851:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 137 -> 129 00000100:00100000:0.0:1788845025.197100:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02a4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641476224:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 1340us (1654us total) trans 4294967400 rc 0/0 00000100:00100000:0.0:1788845025.197109:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 309 00002000:00010000:0.0:1788845025.197113:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845025.197116:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02a4b100 with x1875739641476224 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845025.197123:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02a4b100. 00000100:00080000:1.0:1788845025.197660:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845025.212007:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.212205:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641476352 00000100:00100000:2.0:1788845025.212216:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641476352 00000100:00100000:2.0:1788845025.212279:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 810 00000100:00100000:2.0:1788845025.212289:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641476352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.212309:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.212331:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac670 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fab expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.212415:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.212430:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.212442:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845025.212468:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1002492 00000004:00200000:2.0:1788845025.212472:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845025.212491:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.212515:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac670 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.212548:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.212557:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.212560:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845025.212563:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845025.212628:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641476352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 340us (630us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.212635:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 810 00000100:00100000:2.0:1788845025.230045:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.230135:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641476480 00000100:00100000:3.0:1788845025.230144:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641476480 00000100:00100000:3.0:1788845025.230205:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 811 00000100:00100000:3.0:1788845025.230217:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641476480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.230239:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.230283:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac67e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fb2 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.230322:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.230359:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660216576:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845025.230442:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.233536:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660216576:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845025.233562:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002492 00010000:00010000:3.0:1788845025.233580:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac67e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fb2 expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.233598:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.233601:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845025.233606:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac67e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fb2 expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.233621:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:3.0:1788845025.233685:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641476480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3470us (3648us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845025.233693:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 811 00000100:00100000:0.0:1788845025.235972:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.236232:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641476608 00000100:00100000:2.0:1788845025.236242:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641476608 00000100:00100000:2.0:1788845025.236312:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 529 00000100:00100000:2.0:1788845025.236322:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641476608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.236342:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.236361:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004674de53/0xb2187301287ac685 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fb9 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.236431:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660216704:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.236521:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845025.239677:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660216704:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845025.239718:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004674de53/0xb2187301287ac685 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fb9 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.239744:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004674de53/0xb2187301287ac685 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fb9 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.239757:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0200, rc 0) 00000100:00100000:0.0:1788845025.239844:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641476608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3523us (3880us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.239861:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 529 00000100:00100000:1.0:1788845025.251073:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.251120:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.251222:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641476736 00000100:00100000:2.0:1788845025.251233:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641476736 00000100:00100000:2.0:1788845025.251301:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 812 00000100:00100000:2.0:1788845025.251311:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641476736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.251331:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.251366:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac68c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fc0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.251415:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.251461:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660216832:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.251564:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.252241:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641476864 00000100:00100000:3.0:1788845025.252248:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641476864 00000100:00100000:3.0:1788845025.252305:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 530 00000100:00100000:3.0:1788845025.252312:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641476864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.252323:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.252344:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac693 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fc7 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.252382:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660216960:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845025.252445:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.275272:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660216832:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845025.275299:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002492 00010000:00010000:2.0:1788845025.275318:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac68c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fc0 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.275337:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.275341:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845025.275345:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac68c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fc0 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.275361:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0200, rc 0) 00000100:00100000:2.0:1788845025.275424:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641476736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 24114us (24361us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.275433:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 812 00000100:00100000:3.0:1788845025.275829:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660216960:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845025.275860:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac693 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fc7 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.275877:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ac693 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fc7 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.275924:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:3.0:1788845025.275957:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641476864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 23645us (24836us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845025.275964:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 530 00000100:00100000:1.0:1788845025.287284:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.287397:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641476992 00000100:00100000:2.0:1788845025.287407:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641476992 00000100:00100000:2.0:1788845025.287468:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 813 00000100:00100000:2.0:1788845025.287478:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:14891:x1875739641476992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.287502:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.287539:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac69a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fce expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.287627:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac6a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.287642:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac6a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.287653:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac6a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845025.287680:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1002492 00000004:00200000:2.0:1788845025.287684:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845025.287700:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac6a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.287724:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac69a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fce expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.287741:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac6a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fce expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.287749:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.287752:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845025.287755:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:2.0:1788845025.287814:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641476992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 338us (541us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.287822:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 813 00000100:00100000:1.0:1788845025.293930:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.294043:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641477120 00000100:00100000:0.0:1788845025.294053:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641477120 00000100:00100000:0.0:1788845025.294116:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 814 00000100:00100000:0.0:1788845025.294126:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cb4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:2367:x1875739641477120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.294147:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.294167:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2600/0xb2187301287ac6a8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fd5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.294204:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.294244:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660217088:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.294330:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845025.295293:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.295376:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641477248 00000100:00100000:0.0:1788845025.295384:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641477248 00000100:00100000:0.0:1788845025.295432:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 531 00000100:00100000:0.0:1788845025.295440:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641477248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.295455:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.295468:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac6af lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fdc expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.295526:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660217216:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.295586:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845025.308429:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660217088:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845025.308462:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002492 00010000:00010000:0.0:1788845025.308484:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2600/0xb2187301287ac6a8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fd5 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.308506:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845025.308510:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845025.308515:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2600/0xb2187301287ac6a8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fd5 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.308531:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2600, rc 0) 00000100:00100000:0.0:1788845025.308592:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cb4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:2367:x1875739641477120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 14467us (14673us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.308601:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 814 00000100:00100000:2.0:1788845025.308715:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660217216:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845025.308753:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac6af lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fdc expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.308771:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac6af lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fdc expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.308789:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3c00, rc 0) 00000100:00100000:2.0:1788845025.308815:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641477248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13376us (13526us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.308822:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 531 00000100:00100000:1.0:1788845025.313692:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.313797:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.313877:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641477504 00000100:00100000:0.0:1788845025.313922:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641477504 00000100:00100000:0.0:1788845025.313998:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 532 00000100:00100000:0.0:1788845025.314009:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641477504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.314028:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.314045:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000087ae7707/0xb2187301287ac6b6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fea expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.314103:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660217344:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.314190:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.315700:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641477376 00000100:00100000:2.0:1788845025.315710:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641477376 00000100:00100000:2.0:1788845025.315875:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 815 00000100:00100000:2.0:1788845025.315910:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641477376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.315928:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.315948:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac6bd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fe3 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.315997:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.316033:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660217472:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.316111:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.322869:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660217472:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845025.322928:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002492 00010000:00010000:2.0:1788845025.322964:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac6bd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fe3 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.322999:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.323002:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845025.323008:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac6bd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fe3 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.323023:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0200, rc 0) 00000100:00100000:2.0:1788845025.323072:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641477376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7188us (9394us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.323082:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 815 00000100:00100000:0.0:1788845025.327756:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660217344:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845025.328111:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000087ae7707/0xb2187301287ac6b6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fea expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.328156:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000087ae7707/0xb2187301287ac6b6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fea expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.328171:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3600, rc 0) 00000100:00100000:0.0:1788845025.328253:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641477504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 14245us (14455us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.328262:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 532 00000100:00100000:1.0:1788845025.337338:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.337614:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641477632 00000100:00100000:2.0:1788845025.337626:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641477632 00000100:00100000:2.0:1788845025.337721:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 816 00000100:00100000:2.0:1788845025.337733:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641477632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.337754:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.337784:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ff1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.337814:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.337824:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.337869:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.337944:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 24 pid: 5840 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.337959:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660217600:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845025.338178:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.340229:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660217600:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845025.340268:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.340288:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.340291:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845025.340296:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.340306:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0e00, rc 0) 00000100:00100000:2.0:1788845025.340366:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641477632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2635us (3038us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.340374:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 816 00000100:00100000:1.0:1788845025.342599:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.342694:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641477760 00000100:00100000:2.0:1788845025.342704:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641477760 00010000:00010000:2.0:1788845025.342743:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 25 pid: 5840 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.342793:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 296 00000100:00100000:2.0:1788845025.342803:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2369:x1875739641477760:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845025.342833:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845025.342859:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 25 pid: 5840 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.342877:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 25 pid: 5840 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.343099:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac669 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fa4 expref: 25 pid: 5840 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.343117:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.343123:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.343135:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.343145:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.343184:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.343199:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660217728:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845025.343327:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.349596:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660217728:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845025.349642:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845025.349761:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2369:x1875739641477760:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6957us (7168us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.349936:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 296 00000100:00100000:0.0:1788845025.352164:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.353851:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641477888 00000100:00100000:2.0:1788845025.353940:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641477888 00000100:00100000:2.0:1788845025.354131:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:2.0:1788845025.354142:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641477888:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845025.354361:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002492 -> 1002492 00000004:00010000:2.0:1788845025.354366:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845025 -> 1788845025 00000004:00010000:2.0:1788845025.354369:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845025.354371:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845025 -> 1788845025 00000004:00010000:2.0:1788845025.354373:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845025.354452:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641477888:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 311us (2295us total) trans 4294967629 rc 0/0 00000100:00100000:2.0:1788845025.354460:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:1.0:1788845025.357823:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.358196:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641478016 00000100:00100000:2.0:1788845025.358207:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641478016 00000100:00100000:2.0:1788845025.358330:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 533 00000100:00100000:2.0:1788845025.358341:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641478016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.358361:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.358379:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6cb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7ff8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.358437:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660217856:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.358553:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.361587:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660217856:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845025.361628:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6cb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.361657:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6cb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.361672:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845025.361792:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641478016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3452us (3978us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.361800:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 533 00000100:00100000:1.0:1788845025.376680:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.377074:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641478144 00000100:00100000:3.0:1788845025.377086:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641478144 00000100:00100000:2.0:1788845025.377282:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 534 00000100:00100000:2.0:1788845025.377293:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641478144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.377312:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.377330:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6d2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a7fff expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.377394:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660217984:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.377477:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.380242:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660217984:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845025.380293:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6d2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fff expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.380339:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6d2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fff expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.380355:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845025.380423:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641478144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3131us (3752us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.380433:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 534 00000100:00100000:1.0:1788845025.383657:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.384118:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641478272 00000100:00100000:3.0:1788845025.384129:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641478272 00000100:00100000:3.0:1788845025.384215:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 817 00000100:00100000:3.0:1788845025.384225:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2367:x1875739641478272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.384243:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.384263:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac6d9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8006 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.384319:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.384359:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660218112:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845025.384447:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.392006:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660218112:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845025.392035:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002492 00010000:00010000:3.0:1788845025.392054:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac6d9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8006 expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.392077:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.392080:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845025.392085:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ac6d9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8006 expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.392100:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:3.0:1788845025.392174:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2367:x1875739641478272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7949us (8526us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845025.392183:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 817 00000100:00100000:0.0:1788845025.403260:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.403487:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641478400 00000100:00100000:3.0:1788845025.403497:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641478400 00000100:00100000:3.0:1788845025.403562:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 818 00000100:00100000:3.0:1788845025.403572:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2367:x1875739641478400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.403595:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.403615:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac6e0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a800d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.403669:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.403710:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660218240:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845025.403800:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.408416:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660218240:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845025.408445:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002492 00010000:00010000:3.0:1788845025.408464:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac6e0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a800d expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.408484:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.408486:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845025.408491:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ac6e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a800d expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.408505:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:3.0:1788845025.408568:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2367:x1875739641478400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4996us (5316us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845025.408576:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 818 00000100:00100000:0.0:1788845025.415302:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.415567:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641478528 00000100:00100000:2.0:1788845025.415577:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641478528 00000100:00100000:2.0:1788845025.415637:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 535 00000100:00100000:2.0:1788845025.415648:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641478528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.415668:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.415687:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6e7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8014 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.415743:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660218368:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.416033:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.439250:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660218368:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845025.439298:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6e7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8014 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.439324:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac6e7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8014 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.439361:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845025.439450:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641478528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 23803us (24157us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.439458:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 535 00000100:00100000:1.0:1788845025.454217:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.454332:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641478656 00000100:00100000:0.0:1788845025.454342:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641478656 00000100:00100000:0.0:1788845025.454406:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 536 00000100:00100000:0.0:1788845025.454417:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641478656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.454436:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.454456:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac6ee lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a801b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.454528:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660218496:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.454619:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.459245:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660218496:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845025.459288:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac6ee lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a801b expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.459313:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac6ee lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a801b expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.459326:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3c00, rc 0) 00000100:00100000:2.0:1788845025.459389:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641478656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4974us (5182us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.459397:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 536 00000100:00100000:1.0:1788845025.477048:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.477236:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641478784 00000100:00100000:2.0:1788845025.477246:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641478784 00000100:00100000:2.0:1788845025.477308:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 819 00000100:00100000:2.0:1788845025.477320:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641478784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.477342:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.477365:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac6f5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8022 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.477429:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.477471:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660218624:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.477572:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.479983:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02481500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660218624:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845025.480012:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002492 00010000:00010000:2.0:1788845025.480033:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac6f5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8022 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.480054:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.480058:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845025.480063:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac6f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8022 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.480079:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845025.480276:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641478784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2956us (3238us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.480284:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 819 00000100:00100000:0.0:1788845025.485195:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.485401:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641478912 00000100:00100000:1.0:1788845025.485412:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641478912 00000100:00100000:1.0:1788845025.485475:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 820 00000100:00100000:1.0:1788845025.485487:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:14891:x1875739641478912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845025.485526:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845025.485551:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac6fc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8029 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.485584:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.485614:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac6fc lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8029 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.485652:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.485671:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 25 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.485687:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beab480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660218752:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845025.486393:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.490278:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660219136:0@lo:400:kworker.0 00000100:00100000:3.0:1788845025.490354:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845025.490395:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788845025.490401:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660219008:0@lo:400:kworker.0 00000100:00100000:3.0:1788845025.490425:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845025.490439:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03412d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660219520:0@lo:400:kworker.0 00000100:00100000:3.0:1788845025.490478:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845025.490785:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660219136 00000100:00100000:2.0:1788845025.490794:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660219136 00000100:00100000:2.0:1788845025.490846:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660219008 00000100:00100000:2.0:1788845025.490849:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660219008 00000100:00100000:2.0:1788845025.490866:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:2.0:1788845025.490875:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660219136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845025.490950:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660219136:12345-0@lo:400:kworker.0 Request processed in 74us (595us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.490966:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:2.0:1788845025.490997:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:2.0:1788845025.491003:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660219008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845025.491032:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660219008:12345-0@lo:400:kworker.0 Request processed in 29us (607us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.491037:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:0.0:1788845025.491528:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660218880:0@lo:400:kworker.0 00000100:00100000:0.0:1788845025.491568:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845025.491613:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03413100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660219392:0@lo:400:kworker.0 00000100:00100000:0.0:1788845025.491641:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845025.491660:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788845025.491665:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660219264:0@lo:400:kworker.0 00000100:00100000:0.0:1788845025.491689:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845025.492053:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660218880 00000100:00100000:2.0:1788845025.492059:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660218880 00000100:00080000:1.0:1788845025.492334:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845025.492403:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660219520 00000100:00100000:3.0:1788845025.492409:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660219520 00000100:00100000:3.0:1788845025.492455:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660219264 00000100:00100000:3.0:1788845025.492458:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660219264 00000100:00100000:3.0:1788845025.492482:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:3.0:1788845025.492489:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660219520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845025.492552:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660219520:12345-0@lo:400:kworker.0 Request processed in 63us (2073us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.492558:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:3.0:1788845025.492572:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:3.0:1788845025.492577:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660219264:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845025.492601:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660219264:12345-0@lo:400:kworker.0 Request processed in 25us (912us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.492607:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:3.0:1788845025.492643:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660219008:0@lo:400:kworker.0 00000100:00100000:3.0:1788845025.492653:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03412d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660219520:0@lo:400:kworker.0 00000100:00100000:3.0:1788845025.492662:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660219136:0@lo:400:kworker.0 00000100:00100000:3.0:1788845025.492691:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660219392 00000100:00100000:3.0:1788845025.492693:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660219392 00000100:00100000:3.0:1788845025.492711:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1788845025.492716:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660219392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845025.492846:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1788845025.492865:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660218880:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845025.492917:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660218880:12345-0@lo:400:kworker.0 Request processed in 55us (1348us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.492923:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00080000:1.0:1788845025.493606:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1788845025.493612:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1788845025.493614:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845025.494153:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660219264:0@lo:400:kworker.0 00000100:00100000:0.0:1788845025.494170:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660218880:0@lo:400:kworker.0 00000100:00100000:1.0:1788845025.495841:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.495876:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641479040 00000100:00100000:1.0:1788845025.495898:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641479040 00010000:00010000:1.0:1788845025.495936:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 26 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.495968:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 297 00000100:00100000:1.0:1788845025.495991:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+26:2368:x1875739641479040:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845025.496003:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845025.496029:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 26 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.496063:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 26 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.497440:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac6c4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7ff1 expref: 26 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.497461:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beab480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660218752:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845025.497635:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660219392:12345-0@lo:400:kworker.0 Request processed in 4916us (5991us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.497647:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1788845025.497748:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03413100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660219392:0@lo:400:kworker.0 00010000:00010000:1.0:1788845025.497816:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845025.497826:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac6fc lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8029 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.497839:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac6fc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8029 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.497847:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac6fc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8029 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.502361:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac6fc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8029 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.502388:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845025.502391:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845025.502395:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3800, rc 0) 00000100:00100000:0.0:1788845025.502482:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641478912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16996us (17294us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845025.502492:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 820 00010000:00010000:1.0:1788845025.510295:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac6fc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8029 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.510326:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660219648:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845025.510493:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845025.525471:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.525720:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660219648:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845025.525748:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845025.525808:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641479040:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 29818us (29973us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845025.525821:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 297 00000100:00100000:2.0:1788845025.528699:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641479168 00000100:00100000:2.0:1788845025.528716:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641479168 00002000:00100000:2.0:1788845025.528776:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beab480 x1875739641479168/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845025.528787:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beab480 with x1875739641479168 00000100:00100000:2.0:1788845025.528830:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 487 00000100:00100000:2.0:1788845025.528840:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2369:x1875739641479168:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845025.529702:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2369:x1875739641479168:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 860us (4238us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845025.529713:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 487 00002000:00100000:2.0:1788845025.529721:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beab480 x1875739641479168/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/4536 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845025.529743:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beab480 with x1875739641479168 00000100:00100000:0.0:1788845025.538383:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.538462:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641479296 00000100:00100000:1.0:1788845025.538473:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641479296 00002000:00100000:1.0:1788845025.538588:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb3800 x1875739641479296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845025.538598:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb3800 with x1875739641479296 00000100:00100000:1.0:1788845025.538629:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 488 00000100:00100000:1.0:1788845025.538637:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641479296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845025.538865:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641479296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 229us (491us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845025.538872:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 488 00002000:00100000:1.0:1788845025.538877:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb3800 x1875739641479296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/4536 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845025.538954:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb3800 with x1875739641479296 00000100:00100000:3.0:1788845025.543134:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.543311:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641479424 00000100:00100000:2.0:1788845025.543321:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641479424 00002000:00100000:2.0:1788845025.543372:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413100 x1875739641479424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845025.543383:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413100 with x1875739641479424 00000100:00100000:2.0:1788845025.543421:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 489 00000100:00100000:2.0:1788845025.543429:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641479424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845025.543797:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641479424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 366us (672us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845025.543805:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 489 00002000:00100000:2.0:1788845025.543812:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413100 x1875739641479424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/4536 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845025.543820:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413100 with x1875739641479424 00000100:00100000:0.0:1788845025.564329:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.564592:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641479552 00000100:00100000:1.0:1788845025.564602:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641479552 00002000:00100000:1.0:1788845025.564654:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0000 x1875739641479552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845025.564664:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0000 with x1875739641479552 00000100:00100000:1.0:1788845025.564705:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 490 00000100:00100000:1.0:1788845025.564717:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641479552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845025.565039:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641479552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 325us (720us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845025.565047:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 490 00002000:00100000:1.0:1788845025.565054:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0000 x1875739641479552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/4536 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845025.565062:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0000 with x1875739641479552 00000100:00100000:2.0:1788845025.568697:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.570654:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641479680 00000100:00100000:0.0:1788845025.570667:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641479680 00002000:00100000:2.0:1788845025.574749:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481500 x1875739641479680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845025.574765:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481500 with x1875739641479680 00000100:00100000:2.0:1788845025.574825:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 491 00000100:00100000:2.0:1788845025.574836:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641479680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845025.575589:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641479680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 752us (6899us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845025.575599:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 491 00002000:00100000:2.0:1788845025.575622:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481500 x1875739641479680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/4536 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845025.575630:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481500 with x1875739641479680 00000100:00100000:3.0:1788845025.580674:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.580855:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641479808 00000100:00100000:1.0:1788845025.580866:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641479808 00002000:00100000:1.0:1788845025.580956:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413800 x1875739641479808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845025.580968:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413800 with x1875739641479808 00000100:00100000:1.0:1788845025.581048:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 492 00000100:00100000:1.0:1788845025.581059:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641479808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845025.581923:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641479808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 862us (1258us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845025.581943:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 492 00002000:00100000:1.0:1788845025.581952:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413800 x1875739641479808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/4536 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845025.581961:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413800 with x1875739641479808 00000100:00100000:1.0:1788845025.590440:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.590608:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641479936 00000100:00100000:0.0:1788845025.590640:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641479936 00002000:00100000:0.0:1788845025.590689:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383c380 x1875739641479936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845025.590701:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c380 with x1875739641479936 00000100:00100000:0.0:1788845025.590737:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 493 00000100:00100000:0.0:1788845025.590746:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641479936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845025.591235:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641479936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 487us (802us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845025.591243:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 493 00002000:00100000:0.0:1788845025.591251:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383c380 x1875739641479936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/4536 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845025.591259:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c380 with x1875739641479936 00000100:00100000:2.0:1788845025.594757:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.598745:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641480064 00000100:00100000:3.0:1788845025.598761:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641480064 00002000:00100000:3.0:1788845025.598822:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48e00 x1875739641480064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845025.598834:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48e00 with x1875739641480064 00000100:00100000:3.0:1788845025.598904:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 494 00000100:00100000:3.0:1788845025.598916:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2369:x1875739641480064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845025.601248:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2369:x1875739641480064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 2321us (6489us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845025.601263:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 494 00002000:00100000:3.0:1788845025.601276:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48e00 x1875739641480064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/4536 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845025.601287:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48e00 with x1875739641480064 00000100:00100000:3.0:1788845025.638188:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.638273:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641480192 00000100:00100000:2.0:1788845025.638286:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641480192 00000100:00100000:2.0:1788845025.638357:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 821 00000100:00100000:2.0:1788845025.638368:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641480192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.638395:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.638421:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8030 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.638461:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.638478:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.638493:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.638495:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845025.638500:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:2.0:1788845025.638646:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+26:14891:x1875739641480192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 280us (469us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.638654:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 821 00000100:00100000:1.0:1788845025.654022:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.654293:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641480320 00000100:00100000:0.0:1788845025.654305:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641480320 00002000:00100000:0.0:1788845025.654360:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f774e00 x1875739641480320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845025.654373:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f774e00 with x1875739641480320 00000100:00100000:0.0:1788845025.654421:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 495 00000100:00100000:0.0:1788845025.654431:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641480320:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845025.688670:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641480320:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 34229us (34650us total) trans 0 rc 1003520/1003520 00000100:00100000:0.0:1788845025.688685:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 495 00002000:00100000:0.0:1788845025.688701:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f774e00 x1875739641480320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:131/0 lens 488/440 e 0 to 0 dl 1788845036 ref 1 fl Complete:/600/0 rc 1003520/1003520 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845025.689003:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f774e00 with x1875739641480320 00000100:00100000:3.0:1788845025.692713:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.697663:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641480448 00000100:00100000:0.0:1788845025.697679:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641480448 00000100:00100000:0.0:1788845025.697824:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 537 00000100:00100000:0.0:1788845025.697836:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641480448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.697855:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.697874:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac70a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8037 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.697955:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660219776:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.698126:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00002000:00010000:0.0:1788845025.707651:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845023 -> 1788845025 00000100:00100000:0.0:1788845025.707683:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660219776:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845025.707721:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac70a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8037 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.711372:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac70a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8037 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.711394:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3200, rc 0) 00000100:00100000:0.0:1788845025.711493:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641480448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13658us (18790us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.711792:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 537 00000100:00100000:1.0:1788845025.717765:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.717845:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641480576 00000100:00100000:2.0:1788845025.717857:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641480576 00000100:00100000:2.0:1788845025.718047:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 538 00000100:00100000:2.0:1788845025.718072:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641480576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.718193:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.718214:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac711 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a803e expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.718281:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660219904:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.718387:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.722152:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660219904:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845025.722197:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac711 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a803e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.722225:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac711 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a803e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.722241:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:2.0:1788845025.722375:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641480576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4303us (4618us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.722384:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 538 00000100:00100000:1.0:1788845025.731689:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.731837:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641480704 00000100:00100000:3.0:1788845025.731847:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641480704 00000100:00100000:3.0:1788845025.731931:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:3.0:1788845025.731942:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641480704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.731962:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.731991:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091d1a0ab/0xb2187301287ac718 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8045 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.732071:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660220032:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845025.732235:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.739271:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660220032:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845025.739311:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000091d1a0ab/0xb2187301287ac718 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8045 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.739335:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091d1a0ab/0xb2187301287ac718 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8045 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.739348:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961600, rc 0) 00000100:00100000:3.0:1788845025.739415:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641480704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7475us (7737us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845025.739424:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:1.0:1788845025.745376:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.745468:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641480832 00000100:00100000:0.0:1788845025.745492:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641480832 00000100:00100000:0.0:1788845025.745612:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 540 00000100:00100000:0.0:1788845025.745623:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641480832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.745646:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.745666:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac71f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a804c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.745727:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660220160:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.745815:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845025.748375:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660220160:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845025.748424:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac71f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a804c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.748452:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085baf6c0/0xb2187301287ac71f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a804c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.748466:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3200, rc 0) 00000100:00100000:0.0:1788845025.748655:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641480832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3032us (3287us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.748664:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 540 00000100:00100000:1.0:1788845025.756295:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.756427:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641480960 00000100:00100000:0.0:1788845025.756439:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641480960 00000100:00100000:0.0:1788845025.756500:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 541 00000100:00100000:0.0:1788845025.756522:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641480960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.756539:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.756572:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090de1297/0xb2187301287ac726 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8053 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.756632:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660220288:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.756725:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845025.758768:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660220288:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845025.758811:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090de1297/0xb2187301287ac726 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8053 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.758837:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090de1297/0xb2187301287ac726 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8053 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.758860:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3a00, rc 0) 00000100:00100000:0.0:1788845025.759007:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641480960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2487us (2722us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.759015:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 541 00000100:00100000:1.0:1788845025.765368:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.765448:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641481088 00000100:00100000:2.0:1788845025.765461:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641481088 00000100:00100000:2.0:1788845025.765541:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 542 00000100:00100000:2.0:1788845025.765552:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641481088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.765572:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.765596:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac72d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a805a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.765677:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660220416:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845025.765767:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.775820:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660220416:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845025.775862:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac72d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a805a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.775938:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287ac72d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a805a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.775951:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:2.0:1788845025.776028:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641481088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10477us (10673us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845025.776037:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 542 00000100:00100000:1.0:1788845025.785058:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.785153:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641481344 00000100:00100000:2.0:1788845025.785164:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641481344 00000100:00100000:2.0:1788845025.785236:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 822 00000100:00100000:2.0:1788845025.785249:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641481344:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845025.785313:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845025.785345:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac6a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fce expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.785365:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.785372:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845025.785607:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.785623:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.785632:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.785664:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.785682:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 25 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.785714:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660220544:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845025.785819:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.793196:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660220544:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845025.793226:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.798113:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.798323:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641481472 00000100:00100000:0.0:1788845025.798333:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641481472 00010000:00010000:0.0:1788845025.798375:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 26 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.798421:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 298 00000100:00100000:0.0:1788845025.798430:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641481472:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845025.798445:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845025.798452:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 26 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.798489:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 26 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.798523:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac677 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7fab expref: 26 pid: 5841 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.798538:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845025.798543:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.798551:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.798559:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.798596:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845025.798650:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.798653:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641481472:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 223us (548us total) trans 0 rc 0/0 00010000:00010000:3.0:1788845025.798669:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.798670:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 298 00000001:00080000:3.0:1788845025.798787:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967630 00000004:00010000:3.0:1788845025.798851:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002492 -> 1002492 00000004:00010000:3.0:1788845025.798855:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845025 -> 1788845025 00000004:00010000:3.0:1788845025.798858:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:3.0:1788845025.798861:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845025 -> 1788845025 00000004:00010000:3.0:1788845025.798863:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:3.0:1788845025.798878:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cdcf7d6c reply state 00000000e1877a22 transno 4294967631 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.798922:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845025.799014:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:14891:x1875739641481344:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13768us (13974us total) trans 4294967631 rc 0/0 00000100:00100000:3.0:1788845025.799022:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 822 00000020:00080000:3.0:1788845025.799026:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845025.801233:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.801251:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.801264:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.801269:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c0a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845025.801274:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.801282:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.801297:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.801300:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845025.801304:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.801311:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac734 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.807210:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.807828:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641481600 00000100:00100000:2.0:1788845025.807839:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641481600 00000100:00100000:2.0:1788845025.807919:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 823 00000100:00100000:2.0:1788845025.807931:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:14891:x1875739641481600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845025.807954:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845025.807957:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845025.808015:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac6fc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8029 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.808052:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.808057:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845025.808071:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8061 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.808098:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.808107:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.808138:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.808157:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 24 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845025.808170:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660220672:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845025.808259:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845025.813156:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.813253:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660220672:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845025.813286:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.813305:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845025.813308:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845025.813328:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845025.813339:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:2.0:1788845025.813397:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:14891:x1875739641481600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5469us (6196us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.813406:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 823 00000100:00100000:3.0:1788845025.814008:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641481728 00000100:00100000:3.0:1788845025.814016:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641481728 00010000:00010000:3.0:1788845025.814050:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 25 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.814087:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 299 00000100:00100000:3.0:1788845025.814096:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008914050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641481728:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845025.814109:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845025.814137:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 25 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.814153:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 25 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.814173:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac703 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8030 expref: 25 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.814186:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.814191:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.814200:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.814207:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.814255:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.814267:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660220800:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845025.814345:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.816372:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660220800:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845025.816395:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845025.816445:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008914050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641481728:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2348us (3298us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.816455:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 299 00000100:00100000:2.0:1788845025.819473:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.819539:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.819622:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641481856 00000100:00100000:1.0:1788845025.819634:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641481856 00002000:00010000:1.0:1788845025.819686:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845025.819691:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0e00 with x1875739641481856 00000100:00100000:1.0:1788845025.819732:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 496 00000100:00100000:1.0:1788845025.819743:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:2370:x1875739641481856:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845025.820126:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002492 -> 463737 00000004:00010000:1.0:1788845025.820130:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845025 -> 1788845025 00000004:00010000:1.0:1788845025.820133:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:1.0:1788845025.820135:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845025 -> 1788845025 00000004:00010000:1.0:1788845025.820137:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:1.0:1788845025.820205:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2370:x1875739641481856:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 463us (665us total) trans 4294967632 rc 0/0 00000100:00100000:1.0:1788845025.820214:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 496 00002000:00010000:1.0:1788845025.820218:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845025.820221:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0e00 with x1875739641481856 00000100:00100000:3.0:1788845025.820323:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641481984 00000100:00100000:3.0:1788845025.820329:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641481984 00002000:00010000:3.0:1788845025.820351:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845025.820357:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c1f80 with x1875739641481984 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845025.820365:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c1f80. 00000100:00100000:3.0:1788845025.820383:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 310 00000100:00100000:3.0:1788845025.820391:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641481984:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845025.820535:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1788845025.820539:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845025 -> 1788845025 00002000:00010000:3.0:1788845025.820542:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845025 -> 1788845023 00002000:00010000:3.0:1788845025.820544:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845025 -> 1788845025 00002000:00010000:3.0:1788845025.820547:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:3.0:1788845025.820587:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641481984:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 196us (1124us total) trans 4294967401 rc 0/0 00000100:00100000:3.0:1788845025.820606:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 310 00002000:00010000:3.0:1788845025.820610:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845025.820613:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c1f80 with x1875739641481984 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845025.820618:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c1f80. 00000100:00100000:2.0:1788845025.825592:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.825706:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641482112 00000100:00100000:1.0:1788845025.825716:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641482112 00000100:00100000:1.0:1788845025.825779:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 824 00000100:00100000:1.0:1788845025.825790:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:14891:x1875739641482112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845025.825813:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845025.825834:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2600/0xb2187301287ac742 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8068 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.825990:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.826006:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.826028:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845025.826058:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 463737 00000004:00200000:1.0:1788845025.826062:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845025.826081:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.826107:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2600/0xb2187301287ac742 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 29 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.826125:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 28 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.826134:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845025.826137:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845025.826141:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2800, rc 0) 00000100:00100000:1.0:1788845025.826195:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:14891:x1875739641482112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 407us (612us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845025.826201:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 824 00000100:00100000:0.0:1788845025.833016:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.833139:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641482240 00000100:00100000:3.0:1788845025.833150:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641482240 00000100:00100000:3.0:1788845025.833201:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 825 00000100:00100000:3.0:1788845025.833213:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641482240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.833246:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.833268:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac750 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a806f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.833358:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac757 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.833374:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac757 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.833387:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac757 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845025.833416:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 463737 00000004:00200000:3.0:1788845025.833420:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845025.833438:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac757 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.833465:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac750 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a806f expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.833579:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac757 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a806f expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.833591:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.833594:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845025.833597:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961c00, rc 0) 00000100:00100000:3.0:1788845025.833696:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641482240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 485us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.833704:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 825 00000100:00100000:0.0:1788845025.838850:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.838986:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641482368 00000100:00100000:1.0:1788845025.838997:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641482368 00000100:00100000:1.0:1788845025.839054:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 826 00000100:00100000:1.0:1788845025.839065:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641482368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845025.839085:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845025.839105:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac75e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8076 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.839164:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 27 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.839222:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660220928:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845025.839294:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845025.843066:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.843189:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641482496 00000100:00100000:3.0:1788845025.843199:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641482496 00000100:00100000:3.0:1788845025.843262:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 543 00000100:00100000:3.0:1788845025.843275:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641482496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.843295:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.843315:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287ac765 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a807d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.843374:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660221056:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845025.843458:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845025.843676:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660220928:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845025.843696:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 463737 00010000:00010000:0.0:1788845025.843711:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac75e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8076 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.843747:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845025.843750:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845025.843755:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3c00/0xb2187301287ac75e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8076 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.843771:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3c00, rc 0) 00000100:00100000:0.0:1788845025.843802:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641482368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4739us (4964us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.843809:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 826 00002000:00010000:3.0:1788845025.849085:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1601972 00000100:00100000:3.0:1788845025.849116:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660221056:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845025.849155:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287ac765 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a807d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.849204:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287ac765 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a807d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.849217:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960600, rc 0) 00000100:00100000:3.0:1788845025.849280:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641482496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6008us (6223us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845025.849289:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 543 00000100:00100000:2.0:1788845025.852333:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.852396:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.855290:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641482752 00000100:00100000:0.0:1788845025.855307:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641482752 00000100:00100000:0.0:1788845025.855370:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 544 00000100:00100000:0.0:1788845025.855381:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641482752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.855400:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.855421:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2796fa8/0xb2187301287ac76c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a808b expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.855485:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660221184:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845025.856001:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.857017:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641482624 00000100:00100000:3.0:1788845025.857028:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641482624 00000100:00100000:3.0:1788845025.857080:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 827 00000100:00100000:3.0:1788845025.857090:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641482624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.857108:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.857126:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac773 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8084 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.857160:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 27 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.857193:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660221312:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845025.857249:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845025.860139:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660221184:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845025.860183:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c2796fa8/0xb2187301287ac76c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a808b expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.860208:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2796fa8/0xb2187301287ac76c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a808b expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.860238:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2400, rc 0) 00000100:00100000:0.0:1788845025.860305:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641482752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4924us (7906us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845025.860314:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 544 00000100:00100000:3.0:1788845025.860717:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660221312:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845025.860746:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 463737 00010000:00010000:3.0:1788845025.860759:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac773 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8084 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.860786:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.860789:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845025.860793:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac773 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8084 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.860805:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961600, rc 0) 00000100:00100000:3.0:1788845025.860846:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641482624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3757us (8522us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845025.860853:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 827 00000100:00100000:1.0:1788845025.866067:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.866145:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641483008 00000100:00100000:3.0:1788845025.866155:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641483008 00000100:00100000:3.0:1788845025.866218:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 828 00000100:00100000:3.0:1788845025.866230:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641483008:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845025.866289:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845025.866315:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac757 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a806f expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.866335:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.866340:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845025.866661:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.866677:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 27 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.866687:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.866742:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 27 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.866761:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 27 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.866775:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660221440:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845025.866991:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845025.873013:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660221440:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845025.873047:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.874102:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845025.874367:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.879664:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641483136 00000100:00100000:0.0:1788845025.879678:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641483136 00010000:00010000:0.0:1788845025.879738:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 28 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.879785:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 300 00000100:00100000:0.0:1788845025.879796:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2369:x1875739641483136:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845025.879813:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845025.879821:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 28 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.879841:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 28 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.879877:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2800/0xb2187301287ac749 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8068 expref: 28 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.879922:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845025.879928:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.879938:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.879963:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.880005:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845025.880071:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2369:x1875739641483136:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 276us (5707us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845025.880079:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 300 00010000:00010000:3.0:1788845025.880172:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.880181:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845025.880301:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967633 00000004:00010000:3.0:1788845025.880352:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 463737 -> 463737 00000004:00010000:3.0:1788845025.880357:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845025 -> 1788845025 00000004:00010000:3.0:1788845025.880359:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:3.0:1788845025.880362:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845025 -> 1788845025 00000004:00010000:3.0:1788845025.880364:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:3.0:1788845025.880371:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005c159f9a reply state 00000000e1974db8 transno 4294967634 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.880386:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845025.880461:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641483008:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 14233us (14404us total) trans 4294967634 rc 0/0 00000100:00100000:3.0:1788845025.880475:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 828 00000020:00080000:3.0:1788845025.880479:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788845025.892962:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.893060:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641483264 00000100:00100000:0.0:1788845025.893072:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641483264 00000100:00100000:0.0:1788845025.893142:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 829 00000100:00100000:0.0:1788845025.893152:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641483264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.893177:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.893200:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8092 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.893232:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.893243:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.893278:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.893295:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 26 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.893331:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660221568:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845025.893419:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00010000:00010000:3.0:1788845025.896660:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.896677:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.896685:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.896690:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01960600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845025.896695:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.896703:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.896722:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.896725:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845025.896784:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.896790:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac77a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.912617:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660221568:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845025.912661:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 27 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.912681:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845025.912684:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845025.912689:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 27 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.912700:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2400, rc 0) 00000100:00100000:0.0:1788845025.912770:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641483264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 19619us (19817us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845025.912780:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 829 00000100:00100000:1.0:1788845025.912979:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.915130:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641483392 00000100:00100000:3.0:1788845025.915159:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641483392 00010000:00010000:3.0:1788845025.915208:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 27 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.915261:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 301 00000100:00100000:3.0:1788845025.915271:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2369:x1875739641483392:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845025.915286:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845025.915314:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 27 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.915331:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 27 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.915353:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287ac73b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8061 expref: 27 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.915370:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845025.915376:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.915386:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.915396:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.915440:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.915469:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660221696:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845025.915549:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.937423:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.937538:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660221696:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845025.937564:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845025.937623:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:2369:x1875739641483392:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 22352us (24645us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845025.937634:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 301 00000100:00100000:1.0:1788845025.938185:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641483520 00000100:00100000:1.0:1788845025.938194:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641483520 00000100:00100000:1.0:1788845025.938251:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 545 00000100:00100000:1.0:1788845025.938261:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641483520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845025.938280:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845025.938301:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8099 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.938334:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.938346:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=2s v=(3 12) 00010000:00010000:1.0:1788845025.938351:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:1.0:1788845025.938356:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.938388:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.938421:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 7 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845025.938436:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660221824:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845025.938514:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845025.945192:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660221824:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845025.945235:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.945258:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3e00, rc 0) 00000100:00100000:2.0:1788845025.945273:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845025.945315:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641483520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7054us (7904us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845025.945324:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 545 00000100:00100000:0.0:1788845025.945426:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641483648 00000100:00100000:0.0:1788845025.945433:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641483648 00002000:00100000:0.0:1788845025.945479:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1552384->1601971) req@ffff94a035686300 x1875739641483648/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/0 e 0 to 0 dl 1788845036 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845025.945490:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035686300 with x1875739641483648 ext(1552384->1601971) 00010000:00010000:0.0:1788845025.945498:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 8 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.945531:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 8 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845025.945551:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a035686300 00000100:00100000:0.0:1788845025.945586:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 12 00000100:00100000:0.0:1788845025.945596:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641483648:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845025.955542:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641483648:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 9937us (10259us total) trans 4294967402 rc 0/0 00000100:00100000:1.0:1788845025.955557:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 12 00002000:00100000:1.0:1788845025.955572:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1552384->1601971) req@ffff94a035686300 x1875739641483648/t4294967402(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:131/0 lens 488/448 e 0 to 0 dl 1788845036 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845025.955584:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035686300 with x1875739641483648 ext(1552384->1601971) 00010000:00010000:1.0:1788845025.955597:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 10 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845025.955637:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 10 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845025.955646:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a035686300 00000100:00100000:2.0:1788845025.958430:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.958560:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641483776 00000100:00100000:0.0:1788845025.958570:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641483776 00010000:00010000:0.0:1788845025.958610:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 10 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.958659:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 302 00000100:00100000:0.0:1788845025.958669:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641483776:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845025.958686:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845025.958711:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 10 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.958743:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 10 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.958768:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ac535 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a7e9a expref: 10 pid: 6685 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.958788:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:0.0:1788845025.958792:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:0.0:1788845025.958800:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.958822:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.958861:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845025.958876:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660221952:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845025.959013:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845025.968487:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660221952:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845025.968527:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845025.968597:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641483776:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9928us (10291us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845025.968608:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 302 00000100:00100000:3.0:1788845025.969071:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.969108:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845025.969173:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641483904 00000100:00100000:2.0:1788845025.969180:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641483904 00002000:00010000:2.0:1788845025.969215:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845025.969220:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03411500 with x1875739641483904 00000100:00100000:0.0:1788845025.969250:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641484032 00000100:00100000:2.0:1788845025.969253:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 497 00000100:00100000:0.0:1788845025.969255:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641484032 00000100:00100000:2.0:1788845025.969261:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641483904:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845025.969272:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845025.969276:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03411f80 with x1875739641484032 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845025.969287:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03411f80. 00000100:00100000:0.0:1788845025.969307:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 311 00000100:00100000:0.0:1788845025.969313:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641484032:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:2.0:1788845025.969463:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 463737 -> 642730 00000004:00010000:2.0:1788845025.969467:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845025 -> 1788845025 00000004:00010000:2.0:1788845025.969470:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845025.969472:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845025 -> 1788845025 00000004:00010000:2.0:1788845025.969474:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00002000:00010000:0.0:1788845025.969594:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1601972 -> 0 00002000:00010000:0.0:1788845025.969598:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845025 -> 1788845025 00002000:00010000:0.0:1788845025.969600:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00002000:00010000:0.0:1788845025.969602:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845025 -> 1788845025 00002000:00010000:0.0:1788845025.969604:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:0.0:1788845025.970254:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641484032:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 938us (1142us total) trans 4294967403 rc 0/0 00000100:00100000:0.0:1788845025.970265:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 311 00002000:00010000:0.0:1788845025.970279:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845025.970282:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03411f80 with x1875739641484032 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845025.970288:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03411f80. 00000100:00100000:2.0:1788845025.970589:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641483904:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 703us (894us total) trans 4294967635 rc 0/0 00000100:00100000:2.0:1788845025.970600:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 497 00002000:00010000:2.0:1788845025.970605:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845025.970609:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03411500 with x1875739641483904 00000100:00100000:3.0:1788845025.987233:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845025.987353:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641484160 00000100:00100000:0.0:1788845025.987365:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641484160 00000100:00100000:0.0:1788845025.987434:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 830 00000100:00100000:0.0:1788845025.987445:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641484160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845025.987471:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845025.987491:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac78f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80a0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.987587:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2000/0xb2187301287ac796 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.987604:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2000/0xb2187301287ac796 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.987617:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2000/0xb2187301287ac796 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845025.987647:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 642730 00000004:00200000:0.0:1788845025.987651:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845025.987671:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2000/0xb2187301287ac796 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.987699:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac78f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a0 expref: 30 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.987717:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2000/0xb2187301287ac796 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a0 expref: 29 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845025.987760:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845025.987763:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845025.987766:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2000, rc 0) 00000100:00100000:0.0:1788845025.987831:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:14891:x1875739641484160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 387us (608us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845025.987839:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 830 00000100:00100000:1.0:1788845025.994342:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845025.994437:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641484288 00000100:00100000:3.0:1788845025.994449:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641484288 00000100:00100000:3.0:1788845025.994522:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 831 00000100:00100000:3.0:1788845025.994532:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:14891:x1875739641484288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845025.994554:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845025.994575:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80a7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.994606:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.994616:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.994650:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845025.994682:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 28 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845025.994696:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660222080:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845025.995057:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.002304:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788845026.002325:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660222208:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845026.002394:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845026.002441:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660222208 00000100:00100000:0.0:1788845026.002448:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660222208 00000100:00100000:0.0:1788845026.002496:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 77 00000100:00100000:0.0:1788845026.002505:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660222208:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788845026.002532:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100466688 pending 1703936 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788845026.002577:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660222208:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (182us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845026.002585:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 77 00000100:00100000:0.0:1788845026.002630:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660222208:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845026.004116:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660222080:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845026.004152:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.004172:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.004175:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845026.004183:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.004194:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960600, rc 0) 00000100:00100000:3.0:1788845026.004244:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:14891:x1875739641484288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9714us (9911us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845026.004253:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 831 00000100:00100000:2.0:1788845026.014337:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.014420:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641484416 00000100:00100000:3.0:1788845026.014431:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641484416 00010000:00010000:3.0:1788845026.014472:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 29 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.014522:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 303 00000100:00100000:3.0:1788845026.014533:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641484416:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845026.014563:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845026.014593:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 29 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.014610:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 29 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.014633:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac781 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8092 expref: 29 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.014646:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.014652:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.014663:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.014671:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.014709:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.014723:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660222336:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845026.014807:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845026.017636:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660222336:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845026.017662:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845026.017719:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641484416:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3187us (3390us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.017729:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 303 00000100:00100000:2.0:1788845026.023734:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.023820:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641484544 00000100:00100000:0.0:1788845026.023833:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641484544 00000100:00100000:0.0:1788845026.023936:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 832 00000100:00100000:0.0:1788845026.023949:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641484544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845026.023982:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845026.024005:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7a4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80ae expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.024060:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845026.024110:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660222464:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845026.024202:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000004:00010000:0.0:1788845026.027960:0:5841:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 642730 -> 1006713 00000004:00010000:0.0:1788845026.027980:0:5841:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845025 -> 1788845026 00000004:00010000:0.0:1788845026.027983:0:5841:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845025 -> 1788845026 00000100:00100000:0.0:1788845026.028012:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660222464:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845026.028036:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:0.0:1788845026.028057:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7a4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ae expref: 29 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.028079:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845026.028082:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845026.028087:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7a4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ae expref: 29 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.028102:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2800, rc 0) 00000100:00100000:0.0:1788845026.028162:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641484544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4216us (4439us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845026.028172:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 832 00000100:00100000:1.0:1788845026.039618:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.040069:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641484672 00000100:00100000:0.0:1788845026.040080:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641484672 00000100:00100000:1.0:1788845026.040256:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 833 00000100:00100000:1.0:1788845026.040267:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:2368:x1875739641484672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.040292:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.040314:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2600/0xb2187301287ac7ab lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80b5 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.040353:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.040393:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660222592:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.040484:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.045174:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660222592:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845026.045202:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845026.045222:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2600/0xb2187301287ac7ab lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80b5 expref: 29 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.045244:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.045247:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845026.045268:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2600/0xb2187301287ac7ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80b5 expref: 29 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.045284:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2600, rc 0) 00000100:00100000:1.0:1788845026.045389:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:2368:x1875739641484672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5123us (5782us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.045398:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 833 00000100:00100000:2.0:1788845026.050610:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.051392:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641484800 00000100:00100000:1.0:1788845026.051404:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641484800 00002000:00100000:1.0:1788845026.051461:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686680 x1875739641484800/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.051482:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686680 with x1875739641484800 00000100:00100000:1.0:1788845026.051795:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 498 00000100:00100000:1.0:1788845026.051807:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2369:x1875739641484800:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.052140:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2369:x1875739641484800:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 336us (1541us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845026.052148:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 498 00002000:00100000:1.0:1788845026.052156:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686680 x1875739641484800/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/440 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.052164:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686680 with x1875739641484800 00000100:00100000:2.0:1788845026.054689:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.054774:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641484928 00000100:00100000:3.0:1788845026.054784:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641484928 00000100:00100000:3.0:1788845026.054870:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 834 00000100:00100000:3.0:1788845026.054915:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:14891:x1875739641484928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.054936:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.054958:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac7b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80bc expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.055062:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.055078:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.055091:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845026.055120:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 642730 00000004:00200000:3.0:1788845026.055124:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845026.055142:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.055168:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac7b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.055187:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.055195:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.055199:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845026.055202:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961600, rc 0) 00000100:00100000:3.0:1788845026.055274:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:14891:x1875739641484928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 394us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845026.055289:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 834 00000100:00100000:3.0:1788845026.058539:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.059100:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641485056 00000100:00100000:0.0:1788845026.059110:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641485056 00000100:00100000:0.0:1788845026.059228:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 546 00000100:00100000:0.0:1788845026.059238:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641485056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845026.059258:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845026.059287:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4137750/0xb2187301287ac7c0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80c3 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845026.059350:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660222720:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845026.059440:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00002000:00010000:0.0:1788845026.062201:0:6684:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845025 -> 1788845026 00002000:00010000:0.0:1788845026.062212:0:6684:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845025 -> 1788845026 00000100:00100000:0.0:1788845026.062236:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660222720:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845026.062275:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4137750/0xb2187301287ac7c0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80c3 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.062303:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4137750/0xb2187301287ac7c0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80c3 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.062316:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2800, rc 0) 00000100:00100000:0.0:1788845026.062384:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641485056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3146us (3854us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845026.062393:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 546 00000100:00100000:1.0:1788845026.065137:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.065225:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641485184 00000100:00100000:3.0:1788845026.065238:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641485184 00000100:00100000:3.0:1788845026.065298:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 547 00000100:00100000:3.0:1788845026.065308:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641485184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.065328:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.065346:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac7c7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80ca expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.065417:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660222848:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845026.065512:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845026.069436:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660222848:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845026.069823:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac7c7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ca expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.069861:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac7c7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ca expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.069991:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961c00, rc 0) 00000100:00100000:2.0:1788845026.070105:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641485184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4797us (4976us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845026.070113:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 547 00000100:00100000:2.0:1788845026.074601:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.074717:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641485312 00000100:00100000:0.0:1788845026.074727:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641485312 00000100:00100000:0.0:1788845026.074814:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 548 00000100:00100000:0.0:1788845026.074825:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641485312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845026.074871:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845026.074926:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4137750/0xb2187301287ac7ce lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80d1 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845026.075000:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660222976:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845026.075093:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.077824:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660222976:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845026.077879:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4137750/0xb2187301287ac7ce lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80d1 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.077923:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4137750/0xb2187301287ac7ce lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80d1 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.077937:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2800, rc 0) 00000100:00100000:0.0:1788845026.078009:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641485312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3186us (3419us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845026.078018:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 548 00000100:00100000:3.0:1788845026.083363:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.083500:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641485440 00000100:00100000:1.0:1788845026.083511:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641485440 00002000:00100000:1.0:1788845026.083564:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384b800 x1875739641485440/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.083575:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384b800 with x1875739641485440 00000100:00100000:1.0:1788845026.083636:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 499 00000100:00100000:1.0:1788845026.083646:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2368:x1875739641485440:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.083975:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2368:x1875739641485440:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 320us (614us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845026.083983:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 499 00002000:00100000:1.0:1788845026.083990:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384b800 x1875739641485440/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.083997:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384b800 with x1875739641485440 00000100:00100000:3.0:1788845026.088768:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.089002:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641485568 00000100:00100000:2.0:1788845026.089013:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641485568 00002000:00100000:2.0:1788845026.089064:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f99880 x1875739641485568/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.089076:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f99880 with x1875739641485568 00000100:00100000:2.0:1788845026.089115:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 500 00000100:00100000:2.0:1788845026.089126:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2369:x1875739641485568:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.089400:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2369:x1875739641485568:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (641us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.089408:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 500 00002000:00100000:2.0:1788845026.089415:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f99880 x1875739641485568/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.089424:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f99880 with x1875739641485568 00000100:00100000:3.0:1788845026.093621:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.093732:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641485696 00000100:00100000:1.0:1788845026.093743:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641485696 00002000:00100000:1.0:1788845026.093794:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4e00 x1875739641485696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.093806:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4e00 with x1875739641485696 00000100:00100000:1.0:1788845026.093856:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 501 00000100:00100000:1.0:1788845026.093867:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2367:x1875739641485696:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.094198:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2367:x1875739641485696:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 332us (587us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845026.094207:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 501 00002000:00100000:1.0:1788845026.094215:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4e00 x1875739641485696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.094224:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4e00 with x1875739641485696 00000100:00100000:3.0:1788845026.097838:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.098155:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641485824 00000100:00100000:2.0:1788845026.098165:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641485824 00002000:00100000:2.0:1788845026.098214:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383df80 x1875739641485824/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 520/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.098225:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383df80 with x1875739641485824 00000100:00100000:2.0:1788845026.098280:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 502 00000100:00100000:2.0:1788845026.098290:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2369:x1875739641485824:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845026.103365:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:2.0:1788845026.103374:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845026.103377:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845023 -> 1788845023 00000004:00010000:2.0:1788845026.103379:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845026.103382:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845026.103466:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+30:2369:x1875739641485824:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5172us (5633us total) trans 4294967636 rc 0/0 00000100:00100000:2.0:1788845026.103478:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 502 00002000:00100000:2.0:1788845026.103491:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383df80 x1875739641485824/t4294967636(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 520/456 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.103502:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383df80 with x1875739641485824 00000100:00100000:3.0:1788845026.111368:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.111485:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641485952 00000100:00100000:0.0:1788845026.111495:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641485952 00000100:00100000:0.0:1788845026.111559:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 835 00000100:00100000:0.0:1788845026.111570:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+30:2369:x1875739641485952:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:0.0:1788845026.232363:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+30:2369:x1875739641485952:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 120783us (120995us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845026.232378:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 835 00000001:00080000:3.0:1788845026.233703:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967622 is committed 00000001:00080000:3.0:1788845026.233724:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967621 is committed 00000001:00080000:3.0:1788845026.233759:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967632 is committed 00000001:00080000:3.0:1788845026.233767:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967636 is committed 00000001:00080000:3.0:1788845026.233774:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967635 is committed 00000100:00100000:3.0:1788845026.243620:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.244079:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641486080 00000100:00100000:1.0:1788845026.244089:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641486080 00000100:00100000:1.0:1788845026.244156:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 549 00000100:00100000:1.0:1788845026.244167:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641486080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.244193:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.244213:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac7d5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80d8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.244271:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660223104:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.244364:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.250416:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660223104:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845026.250463:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac7d5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80d8 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.250490:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac7d5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80d8 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.250506:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3600, rc 0) 00000100:00100000:0.0:1788845026.250576:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641486080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6410us (6964us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845026.250586:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 549 00000100:00100000:3.0:1788845026.254648:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.254769:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641486208 00000100:00100000:2.0:1788845026.254780:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641486208 00000100:00100000:2.0:1788845026.254922:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 550 00000100:00100000:2.0:1788845026.254933:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641486208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.254952:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.254983:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac7dc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80df expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.255044:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035686d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660223232:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845026.255119:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.267256:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035686d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660223232:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845026.267302:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac7dc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80df expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.267330:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac7dc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80df expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.267344:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0a00, rc 0) 00000100:00100000:0.0:1788845026.267429:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641486208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12498us (12791us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845026.267438:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 550 00000100:00100000:1.0:1788845026.270717:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.270931:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641486336 00000100:00100000:3.0:1788845026.270940:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641486336 00000100:00100000:3.0:1788845026.271000:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 836 00000100:00100000:3.0:1788845026.271026:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641486336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.271049:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.271069:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac7e3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80e6 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.271107:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.271149:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02381500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660223360:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845026.271292:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b3c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788845026.280101:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845023 -> 1788845026 00000004:00010000:3.0:1788845026.280111:0:11036:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 326 -> 648 00000100:00100000:3.0:1788845026.280139:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02381500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660223360:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845026.280163:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845026.280183:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac7e3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80e6 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.280206:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.280209:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845026.280215:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac7e3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80e6 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.280231:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960200, rc 0) 00000100:00100000:3.0:1788845026.280293:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641486336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9269us (9585us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845026.280302:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 836 00000100:00100000:3.0:1788845026.283621:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.286627:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641486464 00000100:00100000:1.0:1788845026.286642:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641486464 00000100:00100000:1.0:1788845026.287070:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 837 00000100:00100000:1.0:1788845026.287083:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641486464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.287105:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.287129:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80ed expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.287159:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.287170:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.287208:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.287229:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 10 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.287264:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660223488:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845026.287358:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.290505:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.290588:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641486592 00000100:00100000:3.0:1788845026.290598:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641486592 00010000:00010000:3.0:1788845026.290640:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 11 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.290685:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 304 00000100:00100000:3.0:1788845026.290695:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641486592:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845026.290710:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845026.290754:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 11 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.290773:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 11 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.290796:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.290804:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.290815:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.290824:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.290873:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.290910:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02381180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660223616:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845026.291010:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1b3c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788845026.291466:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac79d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a7 expref: 11 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.291498:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660223488:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845026.291525:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.291537:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845026.291540:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845026.291543:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2000, rc 0) 00000100:00100000:2.0:1788845026.291583:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641486464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4504us (7972us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.291592:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 837 00000100:00100000:3.0:1788845026.299452:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.299592:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02381180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660223616:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845026.299619:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845026.299699:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641486592:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9005us (9203us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845026.299713:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 304 00000100:00100000:0.0:1788845026.299777:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641486720 00000100:00100000:0.0:1788845026.299784:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641486720 00002000:00100000:0.0:1788845026.299833:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383480 x1875739641486720/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:132/0 lens 504/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845026.299857:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383480 with x1875739641486720 00000100:00100000:0.0:1788845026.299912:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 503 00000100:00100000:0.0:1788845026.299922:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641486720:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845026.304657:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641486720:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4724us (5202us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1788845026.304671:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 503 00002000:00100000:0.0:1788845026.304686:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383480 x1875739641486720/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:132/0 lens 504/440 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845026.304695:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383480 with x1875739641486720 00000100:00100000:3.0:1788845026.312928:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.314726:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641486848 00000100:00100000:1.0:1788845026.314753:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641486848 00002000:00100000:1.0:1788845026.314815:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02382680 x1875739641486848/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.314826:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02382680 with x1875739641486848 00000100:00100000:1.0:1788845026.314873:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 504 00000100:00100000:1.0:1788845026.314920:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641486848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.315354:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641486848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 469us (2480us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845026.315361:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 504 00002000:00100000:1.0:1788845026.315368:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02382680 x1875739641486848/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.315376:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02382680 with x1875739641486848 00000100:00100000:3.0:1788845026.324783:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.336310:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641486976 00000100:00100000:0.0:1788845026.336328:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641486976 00002000:00100000:0.0:1788845026.336416:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383b80 x1875739641486976/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845026.336427:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383b80 with x1875739641486976 00000100:00100000:0.0:1788845026.336464:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 505 00000100:00100000:0.0:1788845026.336475:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641486976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845026.336725:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641486976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 252us (12004us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845026.336763:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 505 00002000:00100000:0.0:1788845026.336771:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383b80 x1875739641486976/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845026.336779:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383b80 with x1875739641486976 00000100:00100000:3.0:1788845026.342074:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.342246:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641487104 00000100:00100000:2.0:1788845026.342264:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641487104 00002000:00100000:2.0:1788845026.342327:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380700 x1875739641487104/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.342340:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380700 with x1875739641487104 00000100:00100000:2.0:1788845026.342387:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 506 00000100:00100000:2.0:1788845026.342411:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641487104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.342874:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641487104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 463us (811us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.342931:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 506 00002000:00100000:2.0:1788845026.342940:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380700 x1875739641487104/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.342949:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380700 with x1875739641487104 00000100:00100000:3.0:1788845026.356701:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.356830:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641487232 00000100:00100000:0.0:1788845026.356848:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641487232 00000100:00100000:0.0:1788845026.356982:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 838 00000100:00100000:0.0:1788845026.356995:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641487232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845026.357026:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845026.357054:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80f4 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.357094:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.357104:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.357145:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.357167:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 14 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845026.357184:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660223744:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845026.357289:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.361355:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660223744:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845026.361409:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.361434:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845026.361437:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845026.361443:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.361457:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2800, rc 0) 00000100:00100000:0.0:1788845026.361543:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641487232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4551us (4858us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845026.361554:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 838 00000100:00100000:1.0:1788845026.364537:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.365102:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641487360 00000100:00100000:3.0:1788845026.365116:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641487360 00010000:00010000:3.0:1788845026.365170:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.365227:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 305 00000100:00100000:3.0:1788845026.365237:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641487360:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845026.365261:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845026.365296:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.365317:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.365344:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2000/0xb2187301287ac7ea lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80ed expref: 15 pid: 5840 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.365364:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.365371:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.365410:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.365420:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.365484:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.365502:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660223872:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845026.366345:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.377783:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.378521:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660223872:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845026.378555:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845026.378746:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641487360:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 13508us (14222us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845026.378760:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 305 00000100:00100000:2.0:1788845026.379715:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641487488 00000100:00100000:2.0:1788845026.379729:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641487488 00000100:00100000:2.0:1788845026.379823:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 507 00000100:00100000:2.0:1788845026.380035:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641487488:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845026.381938:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:2.0:1788845026.381950:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845026.381954:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845026.381967:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845026.381970:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:2.0:1788845026.382098:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641487488:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 2258us (4324us total) trans 4294967637 rc 0/0 00000100:00100000:2.0:1788845026.382112:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 507 00000100:00100000:0.0:1788845026.402171:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.404651:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641487616 00000100:00100000:2.0:1788845026.404678:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641487616 00000100:00100000:2.0:1788845026.404745:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 551 00000100:00100000:2.0:1788845026.404756:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641487616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.404776:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.404819:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004674de53/0xb2187301287ac7f8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a80fb expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.405080:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660224000:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845026.405159:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00002000:00010000:3.0:1788845026.420338:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845023 -> 1788845026 00000100:00100000:3.0:1788845026.420420:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660224000:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845026.420465:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004674de53/0xb2187301287ac7f8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80fb expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.420492:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004674de53/0xb2187301287ac7f8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80fb expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.420529:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0200, rc 0) 00000100:00100000:3.0:1788845026.420599:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641487616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15844us (18444us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845026.420608:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 551 00000100:00100000:3.0:1788845026.440613:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.440855:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641487744 00000100:00100000:0.0:1788845026.440869:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641487744 00000100:00100000:0.0:1788845026.440986:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:0.0:1788845026.440999:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641487744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845026.441020:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845026.441040:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f15a0e2e/0xb2187301287ac7ff lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8102 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.441269:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660224128:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.441375:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.444212:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660224128:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845026.444272:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f15a0e2e/0xb2187301287ac7ff lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8102 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.444322:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f15a0e2e/0xb2187301287ac7ff lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8102 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.444336:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3000, rc 0) 00000100:00100000:1.0:1788845026.444404:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641487744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3407us (3802us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.444412:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:3.0:1788845026.451291:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.454501:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641488000 00000100:00100000:0.0:1788845026.454515:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641488000 00000100:00100000:0.0:1788845026.454602:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 839 00000100:00100000:0.0:1788845026.454615:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641488000:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845026.454688:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845026.454717:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2000/0xb2187301287ac796 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80a0 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.454737:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845026.454743:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845026.455033:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.455051:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.455060:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.455097:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.455116:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 11 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845026.455130:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8e680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660224256:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845026.455226:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.463494:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.463593:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8e680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660224256:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845026.463627:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.463727:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641488128 00000100:00100000:1.0:1788845026.463735:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641488128 00010000:00010000:1.0:1788845026.463777:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 12 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.463821:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 306 00000100:00100000:1.0:1788845026.463832:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008917450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641488128:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845026.463846:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845026.463854:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 12 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.463871:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 12 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.463950:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac7b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80bc expref: 12 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.463981:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.464035:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.464045:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.464054:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.464082:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845026.464154:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008917450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641488128:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 323us (687us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845026.464162:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 306 00010000:00010000:3.0:1788845026.467380:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.467403:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845026.468622:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967638 00000004:00010000:3.0:1788845026.468719:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:3.0:1788845026.468724:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845026 -> 1788845026 00000004:00010000:3.0:1788845026.468727:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:3.0:1788845026.468730:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845026 -> 1788845026 00000004:00010000:3.0:1788845026.468732:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000004:00010000:3.0:1788845026.468744:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b912c8af reply state 00000000e7a7bfd8 transno 4294967639 ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.468757:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845026.468937:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641488000:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 14320us (17650us total) trans 4294967639 rc 0/0 00000100:00100000:3.0:1788845026.468949:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 839 00000020:00080000:3.0:1788845026.468985:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788845026.482808:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845026.482967:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.482974:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641488256 00010000:00010000:3.0:1788845026.482984:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.482986:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641488256 00010000:00010000:3.0:1788845026.482995:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.483002:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f028c3200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845026.483006:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.483015:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.483042:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.483046:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845026.483052:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.483059:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3200/0xb2187301287ac806 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.483073:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 840 00000100:00100000:2.0:1788845026.483085:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641488256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.483121:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.483138:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8109 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.483164:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.483173:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.483207:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.483226:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 10 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.483240:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660224384:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845026.483315:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.488585:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.488778:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641488384 00000100:00100000:3.0:1788845026.488787:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641488384 00010000:00010000:3.0:1788845026.488879:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 11 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.488965:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 307 00000100:00100000:3.0:1788845026.488976:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641488384:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845026.488997:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845026.489022:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 11 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.489039:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 11 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.489061:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.489068:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.489079:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.489088:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.489124:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.489138:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660224512:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845026.489210:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00010000:00010000:2.0:1788845026.491675:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2800/0xb2187301287ac7f1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a80f4 expref: 11 pid: 5841 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.491705:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660224384:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845026.491737:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.491754:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845026.491773:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845026.491777:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0e00, rc 0) 00000100:00100000:2.0:1788845026.493847:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641488256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10745us (11028us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.493859:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 840 00000100:00100000:3.0:1788845026.498597:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660224512:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845026.498625:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845026.498688:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641488384:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9713us (10113us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845026.498716:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 307 00000100:00100000:0.0:1788845026.511730:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.512179:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641488512 00000100:00100000:1.0:1788845026.512190:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641488512 00002000:00010000:1.0:1788845026.512232:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1439494->18446744073709551615) 00002000:00100000:1.0:1788845026.512237:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8fb80 with x1875739641488512 ext(1439494->18446744073709551615) 00002000:00010000:1.0:1788845026.512249:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8fb80. 00000100:00100000:1.0:1788845026.512312:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 312 00000100:00100000:1.0:1788845026.512322:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641488512:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845026.512564:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1439494 00002000:00010000:1.0:1788845026.512567:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845026 -> 1788845026 00002000:00010000:1.0:1788845026.512570:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845026 -> 1788845023 00002000:00010000:1.0:1788845026.512572:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845026 -> 1788845026 00002000:00010000:1.0:1788845026.512575:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:1.0:1788845026.512632:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641488512:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 311us (910us total) trans 4294967404 rc 0/0 00000100:00100000:1.0:1788845026.512656:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 312 00002000:00010000:1.0:1788845026.512659:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1439494->18446744073709551615) 00002000:00100000:1.0:1788845026.512662:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8fb80 with x1875739641488512 ext(1439494->18446744073709551615) 00002000:00010000:1.0:1788845026.512667:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8fb80. 00000100:00100000:0.0:1788845026.519576:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.519693:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641488640 00000100:00100000:2.0:1788845026.519704:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641488640 00000100:00100000:2.0:1788845026.519809:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 841 00000100:00100000:2.0:1788845026.519821:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641488640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.519996:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.520020:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac814 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8110 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.520116:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac81b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.520131:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac81b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.520142:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac81b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845026.520169:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1439494 00000004:00200000:2.0:1788845026.520173:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845026.520191:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac81b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.520220:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac814 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8110 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.520261:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac81b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8110 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.520270:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845026.520273:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845026.520276:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0200, rc 0) 00000100:00100000:2.0:1788845026.520351:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641488640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 532us (785us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.520367:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 841 00000100:00100000:0.0:1788845026.526558:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.526596:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.526755:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641488768 00000100:00100000:1.0:1788845026.526765:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641488768 00000100:00100000:1.0:1788845026.526833:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 842 00000100:00100000:1.0:1788845026.526841:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641488768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.526859:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.526877:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac822 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8117 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.526936:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.527006:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660224640:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.527077:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845026.530624:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641488896 00000100:00100000:2.0:1788845026.530639:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641488896 00000100:00100000:2.0:1788845026.530739:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 553 00000100:00100000:2.0:1788845026.530750:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641488896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.530768:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.530785:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac829 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a811e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.530863:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035684a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660224768:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845026.530976:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00002000:00010000:2.0:1788845026.536631:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845023 -> 1788845026 00000100:00100000:2.0:1788845026.536665:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035684a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660224768:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845026.536703:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac829 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a811e expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.536749:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac829 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a811e expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.536763:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845026.536840:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641488896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6091us (10241us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845026.536850:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 553 00000100:00100000:1.0:1788845026.539785:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660224640:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845026.539812:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845026.539943:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac822 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8117 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.539973:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.539977:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845026.539982:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac822 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8117 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.540019:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3000, rc 0) 00000100:00100000:1.0:1788845026.540362:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641488768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13520us (13813us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.540370:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 842 00000100:00100000:0.0:1788845026.547417:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.547452:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.547461:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641489152 00000100:00100000:1.0:1788845026.547470:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641489152 00000100:00100000:3.0:1788845026.547543:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641489024 00000100:00100000:3.0:1788845026.547548:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641489024 00000100:00100000:1.0:1788845026.547583:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 554 00000100:00100000:3.0:1788845026.547586:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 843 00000100:00100000:3.0:1788845026.547594:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641489024:12345-192.168.201.29@tcp:101:fsx.0 00000100:00100000:1.0:1788845026.547594:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641489152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.547612:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.547612:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.547630:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a35ea6c/0xb2187301287ac837 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a812c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.547631:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac830 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8125 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.547663:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.547684:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660225024:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.547684:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660224896:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845026.547853:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.548080:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.550388:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660224896:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845026.550421:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a35ea6c/0xb2187301287ac837 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a812c expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.550440:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a35ea6c/0xb2187301287ac837 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a812c expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.550450:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2200, rc 0) 00000100:00100000:1.0:1788845026.550500:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641489152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2907us (3091us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.550523:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 554 00000100:00100000:3.0:1788845026.552502:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660225024:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845026.552529:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845026.552545:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac830 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8125 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.552565:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.552568:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845026.552573:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac830 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8125 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.552694:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960200, rc 0) 00000100:00100000:3.0:1788845026.552741:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641489024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5148us (5287us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845026.552766:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 843 00000100:00100000:0.0:1788845026.555401:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.555443:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.555444:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641489408 00000100:00100000:3.0:1788845026.555457:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641489408 00000100:00100000:3.0:1788845026.555509:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 555 00000100:00100000:3.0:1788845026.555519:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641489408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.555537:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.555555:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008edd7da8/0xb2187301287ac83e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a813a expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.555612:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660225152:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845026.555707:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641489280 00000100:00100000:2.0:1788845026.555711:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641489280 00000100:00100000:2.0:1788845026.555752:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 844 00000100:00100000:2.0:1788845026.555758:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641489280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.555768:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.555779:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac845 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8133 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.555864:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.555908:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660225280:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845026.556089:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845026.556120:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.558418:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660225152:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845026.558458:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008edd7da8/0xb2187301287ac83e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a813a expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.558482:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008edd7da8/0xb2187301287ac83e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a813a expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.558496:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960c00, rc 0) 00000100:00100000:3.0:1788845026.558580:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641489408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3062us (3187us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845026.558587:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 555 00000100:00100000:2.0:1788845026.558866:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660225280:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845026.558911:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845026.558925:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac845 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8133 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.558969:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845026.558972:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845026.558977:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287ac845 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8133 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.558988:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845026.559015:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641489280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3256us (3571us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845026.559022:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 844 00000100:00100000:1.0:1788845026.561543:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.561617:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641489536 00000100:00100000:2.0:1788845026.561628:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641489536 00000100:00100000:2.0:1788845026.561721:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 556 00000100:00100000:2.0:1788845026.561732:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641489536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.561750:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.561769:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8141 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.561802:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.561810:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:2.0:1788845026.561814:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:2.0:1788845026.561819:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1101824->1105919] (req 1101824->1105919) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.561871:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.561948:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 7 pid: 7983 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.561976:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035684a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660225408:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845026.562052:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00938e000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.564784:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.564855:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641489664 00000100:00100000:1.0:1788845026.564864:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641489664 00010000:00010000:1.0:1788845026.564925:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 8 pid: 7983 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.564950:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 308 00000100:00100000:1.0:1788845026.564968:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641489664:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845026.564984:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845026.565013:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 8 pid: 7983 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.565032:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 8 pid: 7983 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.565057:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:1.0:1788845026.565062:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:1.0:1788845026.565069:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.565080:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.565117:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.565146:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660225536:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845026.565223:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00010000:00010000:2.0:1788845026.565700:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b93269b/0xb2187301287ac788 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8099 expref: 8 pid: 7983 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.565718:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035684a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660225408:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845026.565743:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.565754:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845026.565799:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641489536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4067us (4266us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.565807:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 556 00000100:00100000:1.0:1788845026.568203:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660225536:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845026.568232:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845026.568522:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641489664:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3562us (3744us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845026.568533:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 308 00000100:00100000:0.0:1788845026.571689:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.571779:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641489792 00000100:00100000:1.0:1788845026.571787:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641489792 00002000:00100000:1.0:1788845026.572054:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1101824->1105919) req@ffff949f01d8df80 x1875739641489792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.572062:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8df80 with x1875739641489792 ext(1101824->1105919) 00002000:00010000:1.0:1788845026.572067:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8df80 00000100:00100000:1.0:1788845026.572129:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 313 00000100:00100000:1.0:1788845026.572139:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641489792:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.572462:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641489792:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 322us (778us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845026.572470:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 313 00002000:00100000:1.0:1788845026.572487:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1101824->1105919) req@ffff949f01d8df80 x1875739641489792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.572496:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8df80 with x1875739641489792 ext(1101824->1105919) 00002000:00010000:1.0:1788845026.572500:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8df80 00000100:00100000:0.0:1788845026.575487:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.575528:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641489920 00000100:00100000:2.0:1788845026.575538:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641489920 00002000:00100000:2.0:1788845026.575582:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1105920->1110015) req@ffff949f01d8c380 x1875739641489920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.575613:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8c380 with x1875739641489920 ext(1105920->1110015) 00002000:00010000:2.0:1788845026.575618:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8c380 00000100:00100000:2.0:1788845026.575650:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 314 00000100:00100000:2.0:1788845026.575660:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641489920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.575953:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641489920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (483us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.575973:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 314 00002000:00100000:2.0:1788845026.575981:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1105920->1110015) req@ffff949f01d8c380 x1875739641489920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.575989:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8c380 with x1875739641489920 ext(1105920->1110015) 00002000:00010000:2.0:1788845026.575993:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8c380 00000100:00100000:0.0:1788845026.578344:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.578438:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641490048 00000100:00100000:1.0:1788845026.578449:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641490048 00002000:00100000:1.0:1788845026.578494:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1110016->1114111) req@ffff949f01d8f800 x1875739641490048/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.578504:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8f800 with x1875739641490048 ext(1110016->1114111) 00002000:00010000:1.0:1788845026.578511:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8f800 00000100:00100000:1.0:1788845026.578541:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 315 00000100:00100000:1.0:1788845026.578550:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641490048:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.578758:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641490048:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 209us (426us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845026.578763:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 315 00002000:00100000:1.0:1788845026.578769:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1110016->1114111) req@ffff949f01d8f800 x1875739641490048/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.578774:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8f800 with x1875739641490048 ext(1110016->1114111) 00002000:00010000:1.0:1788845026.578776:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8f800 00000100:00100000:0.0:1788845026.581938:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.582754:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641490176 00000100:00100000:2.0:1788845026.582765:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641490176 00002000:00100000:2.0:1788845026.582818:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1114112->1118207) req@ffff949f01d8dc00 x1875739641490176/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.582827:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8dc00 with x1875739641490176 ext(1114112->1118207) 00002000:00010000:2.0:1788845026.582833:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8dc00 00000100:00100000:2.0:1788845026.582868:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 316 00000100:00100000:2.0:1788845026.582878:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641490176:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.583114:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641490176:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 237us (1184us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.583133:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 316 00002000:00100000:2.0:1788845026.583154:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1114112->1118207) req@ffff949f01d8dc00 x1875739641490176/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.583161:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8dc00 with x1875739641490176 ext(1114112->1118207) 00002000:00010000:2.0:1788845026.583165:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8dc00 00000100:00100000:0.0:1788845026.586516:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.586720:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641490304 00000100:00100000:3.0:1788845026.586729:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641490304 00002000:00100000:3.0:1788845026.586782:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1118208->1122303) req@ffff949f01d8e300 x1875739641490304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.586791:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8e300 with x1875739641490304 ext(1118208->1122303) 00002000:00010000:3.0:1788845026.586796:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8e300 00000100:00100000:3.0:1788845026.586871:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 317 00000100:00100000:3.0:1788845026.586878:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641490304:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845026.587220:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641490304:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 341us (713us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845026.587229:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 317 00002000:00100000:3.0:1788845026.587236:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1118208->1122303) req@ffff949f01d8e300 x1875739641490304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.587242:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8e300 with x1875739641490304 ext(1118208->1122303) 00002000:00010000:3.0:1788845026.587244:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8e300 00000100:00100000:0.0:1788845026.589743:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.589810:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641490432 00000100:00100000:2.0:1788845026.589818:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641490432 00002000:00100000:2.0:1788845026.589861:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1122304->1126399) req@ffff949f01d8d500 x1875739641490432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.589872:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8d500 with x1875739641490432 ext(1122304->1126399) 00002000:00010000:2.0:1788845026.589879:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8d500 00000100:00100000:2.0:1788845026.589927:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 318 00000100:00100000:2.0:1788845026.589937:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641490432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.590150:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641490432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 214us (413us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.590158:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 318 00002000:00100000:2.0:1788845026.590165:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1122304->1126399) req@ffff949f01d8d500 x1875739641490432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.590172:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8d500 with x1875739641490432 ext(1122304->1126399) 00002000:00010000:2.0:1788845026.590592:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8d500 00000100:00100000:0.0:1788845026.592286:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.592446:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641490560 00000100:00100000:3.0:1788845026.592454:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641490560 00002000:00100000:3.0:1788845026.592539:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1126400->1130495) req@ffff949f01d8f480 x1875739641490560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.592547:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8f480 with x1875739641490560 ext(1126400->1130495) 00002000:00010000:3.0:1788845026.592551:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8f480 00000100:00100000:3.0:1788845026.592607:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 319 00000100:00100000:3.0:1788845026.592614:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641490560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845026.592785:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641490560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 172us (505us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845026.592791:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 319 00002000:00100000:3.0:1788845026.592814:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1126400->1130495) req@ffff949f01d8f480 x1875739641490560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.592821:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8f480 with x1875739641490560 ext(1126400->1130495) 00002000:00010000:3.0:1788845026.592824:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8f480 00000100:00100000:0.0:1788845026.595629:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.595724:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641490688 00000100:00100000:2.0:1788845026.595734:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641490688 00002000:00100000:2.0:1788845026.595778:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1130496->1134591) req@ffff949f01964380 x1875739641490688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.595789:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01964380 with x1875739641490688 ext(1130496->1134591) 00002000:00010000:2.0:1788845026.595802:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01964380 00000100:00100000:2.0:1788845026.595969:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 320 00000100:00100000:2.0:1788845026.595980:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641490688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.596212:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641490688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (591us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.596219:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 320 00002000:00100000:2.0:1788845026.596227:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1130496->1134591) req@ffff949f01964380 x1875739641490688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.596234:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01964380 with x1875739641490688 ext(1130496->1134591) 00002000:00010000:2.0:1788845026.596238:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01964380 00000100:00100000:0.0:1788845026.598992:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.599071:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641490816 00000100:00100000:3.0:1788845026.599079:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641490816 00002000:00100000:3.0:1788845026.599118:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1134592->1138687) req@ffff949f01967800 x1875739641490816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.599126:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01967800 with x1875739641490816 ext(1134592->1138687) 00002000:00010000:3.0:1788845026.599130:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01967800 00000100:00100000:3.0:1788845026.599154:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 321 00000100:00100000:3.0:1788845026.599163:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641490816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845026.599409:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641490816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 247us (429us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845026.599415:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 321 00002000:00100000:3.0:1788845026.599422:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1134592->1138687) req@ffff949f01967800 x1875739641490816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.599431:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01967800 with x1875739641490816 ext(1134592->1138687) 00002000:00010000:3.0:1788845026.599434:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01967800 00000100:00100000:0.0:1788845026.602065:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.602219:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641490944 00000100:00100000:2.0:1788845026.602229:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641490944 00002000:00100000:2.0:1788845026.602278:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1138688->1142783) req@ffff949f01965c00 x1875739641490944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.602289:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965c00 with x1875739641490944 ext(1138688->1142783) 00002000:00010000:2.0:1788845026.602295:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965c00 00000100:00100000:2.0:1788845026.602331:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 322 00000100:00100000:2.0:1788845026.602341:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641490944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.602620:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641490944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 279us (564us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.602628:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 322 00002000:00100000:2.0:1788845026.602634:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1138688->1142783) req@ffff949f01965c00 x1875739641490944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.602662:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965c00 with x1875739641490944 ext(1138688->1142783) 00002000:00010000:2.0:1788845026.602666:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965c00 00000100:00100000:0.0:1788845026.606223:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.606311:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641491072 00000100:00100000:3.0:1788845026.606322:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641491072 00002000:00100000:3.0:1788845026.606375:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1142784->1146879) req@ffff949f01966300 x1875739641491072/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.606387:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01966300 with x1875739641491072 ext(1142784->1146879) 00002000:00010000:3.0:1788845026.606410:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01966300 00000100:00100000:3.0:1788845026.606448:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 323 00000100:00100000:3.0:1788845026.606458:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641491072:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845026.606696:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641491072:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 238us (482us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845026.606704:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 323 00002000:00100000:3.0:1788845026.606712:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1142784->1146879) req@ffff949f01966300 x1875739641491072/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.606720:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01966300 with x1875739641491072 ext(1142784->1146879) 00002000:00010000:3.0:1788845026.606724:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01966300 00000100:00100000:0.0:1788845026.610520:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.610621:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641491200 00000100:00100000:1.0:1788845026.610631:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641491200 00002000:00100000:1.0:1788845026.610680:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1146880->1150975) req@ffff949f01965500 x1875739641491200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.610692:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965500 with x1875739641491200 ext(1146880->1150975) 00002000:00010000:1.0:1788845026.610699:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965500 00000100:00100000:1.0:1788845026.610735:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 324 00000100:00100000:1.0:1788845026.610745:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641491200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.611040:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641491200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 295us (528us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845026.611062:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 324 00002000:00100000:1.0:1788845026.611069:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1146880->1150975) req@ffff949f01965500 x1875739641491200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.611078:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965500 with x1875739641491200 ext(1146880->1150975) 00002000:00010000:1.0:1788845026.611081:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965500 00000100:00100000:0.0:1788845026.614512:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.614655:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641491328 00000100:00100000:2.0:1788845026.614667:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641491328 00002000:00100000:2.0:1788845026.614725:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1150976->1155071) req@ffff949f01965180 x1875739641491328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.614737:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965180 with x1875739641491328 ext(1150976->1155071) 00002000:00010000:2.0:1788845026.614744:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965180 00000100:00100000:2.0:1788845026.614782:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 325 00000100:00100000:2.0:1788845026.614793:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641491328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.615075:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641491328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 283us (576us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.615083:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 325 00002000:00100000:2.0:1788845026.615090:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1150976->1155071) req@ffff949f01965180 x1875739641491328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.615097:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965180 with x1875739641491328 ext(1150976->1155071) 00002000:00010000:2.0:1788845026.615101:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965180 00000100:00100000:0.0:1788845026.618784:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.618920:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641491456 00000100:00100000:3.0:1788845026.618931:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641491456 00002000:00100000:3.0:1788845026.618991:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1155072->1159167) req@ffff949f01967480 x1875739641491456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.619004:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01967480 with x1875739641491456 ext(1155072->1159167) 00002000:00010000:3.0:1788845026.619010:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01967480 00000100:00100000:3.0:1788845026.619049:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 326 00000100:00100000:3.0:1788845026.619059:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641491456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845026.619281:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641491456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 223us (505us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845026.619290:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 326 00002000:00100000:3.0:1788845026.619296:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1155072->1159167) req@ffff949f01967480 x1875739641491456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.619326:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01967480 with x1875739641491456 ext(1155072->1159167) 00002000:00010000:3.0:1788845026.619330:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01967480 00000100:00100000:0.0:1788845026.622653:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.622752:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641491584 00000100:00100000:1.0:1788845026.622762:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641491584 00002000:00100000:1.0:1788845026.622809:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1159168->1163263) req@ffff949f01966680 x1875739641491584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.622820:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01966680 with x1875739641491584 ext(1159168->1163263) 00002000:00010000:1.0:1788845026.622825:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01966680 00000100:00100000:1.0:1788845026.622870:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 327 00000100:00100000:1.0:1788845026.622917:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641491584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.623155:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641491584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 275us (511us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845026.623163:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 327 00002000:00100000:1.0:1788845026.623170:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1159168->1163263) req@ffff949f01966680 x1875739641491584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.623178:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01966680 with x1875739641491584 ext(1159168->1163263) 00002000:00010000:1.0:1788845026.623182:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01966680 00000100:00100000:0.0:1788845026.633811:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.634328:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641491712 00000100:00100000:1.0:1788845026.634339:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641491712 00002000:00100000:1.0:1788845026.634396:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1163264->1167359) req@ffff949f01967b80 x1875739641491712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.634438:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01967b80 with x1875739641491712 ext(1163264->1167359) 00002000:00010000:1.0:1788845026.634444:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01967b80 00000100:00100000:1.0:1788845026.634484:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 328 00000100:00100000:1.0:1788845026.634495:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641491712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.635464:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641491712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 966us (1657us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845026.635482:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 328 00002000:00100000:1.0:1788845026.635492:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1163264->1167359) req@ffff949f01967b80 x1875739641491712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.635501:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01967b80 with x1875739641491712 ext(1163264->1167359) 00002000:00010000:1.0:1788845026.635505:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01967b80 00000100:00100000:0.0:1788845026.647944:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.648084:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641491840 00000100:00100000:3.0:1788845026.648094:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641491840 00002000:00100000:3.0:1788845026.648287:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1167360->1171455) req@ffff949f01964700 x1875739641491840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845026.648300:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01964700 with x1875739641491840 ext(1167360->1171455) 00002000:00010000:3.0:1788845026.648306:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01964700 00000100:00100000:2.0:1788845026.648600:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 329 00000100:00100000:2.0:1788845026.648614:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641491840:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.648844:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641491840:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (911us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845026.648852:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 329 00002000:00100000:2.0:1788845026.648873:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1167360->1171455) req@ffff949f01964700 x1875739641491840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/4536 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.648898:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01964700 with x1875739641491840 ext(1167360->1171455) 00002000:00010000:2.0:1788845026.648902:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01964700 00000100:00100000:2.0:1788845026.655550:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.657562:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641491968 00000100:00100000:1.0:1788845026.657576:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641491968 00000100:00100000:1.0:1788845026.657721:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 845 00000100:00100000:1.0:1788845026.657732:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641491968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.657754:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.657776:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac853 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8148 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.657816:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.657868:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660225664:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.658023:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.662765:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660225664:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845026.662800:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845026.662823:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac853 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8148 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.662949:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.662964:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845026.662970:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac853 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8148 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.662986:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2200, rc 0) 00000100:00100000:1.0:1788845026.663109:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641491968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5378us (7570us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.663126:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 845 00000100:00100000:0.0:1788845026.665451:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.665574:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641492096 00000100:00100000:1.0:1788845026.665584:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641492096 00000100:00100000:1.0:1788845026.665695:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 846 00000100:00100000:1.0:1788845026.665705:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641492096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.665727:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.665750:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac85a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a814f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.665787:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.665842:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660225792:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.665970:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.674010:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660225792:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845026.674042:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845026.674062:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac85a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a814f expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.674084:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.674088:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845026.674092:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac85a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a814f expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.674107:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3000, rc 0) 00000100:00100000:1.0:1788845026.674302:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641492096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8596us (8859us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.674313:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 846 00000100:00100000:0.0:1788845026.682587:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.683308:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641492224 00000100:00100000:3.0:1788845026.683321:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641492224 00000100:00100000:3.0:1788845026.683393:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 847 00000100:00100000:3.0:1788845026.683405:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641492224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.683428:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.683454:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8156 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.683488:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.683499:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.683535:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.683553:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 15 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.683578:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660225920:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845026.683682:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.689139:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660225920:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845026.689177:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.689198:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.689201:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845026.689205:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.689217:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960200, rc 0) 00000100:00100000:3.0:1788845026.689331:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641492224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5928us (6755us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845026.689341:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 847 00000100:00100000:2.0:1788845026.697044:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.697154:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641492352 00000100:00100000:1.0:1788845026.697164:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641492352 00010000:00010000:1.0:1788845026.697211:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.697256:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 309 00000100:00100000:1.0:1788845026.697267:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641492352:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845026.697284:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845026.697311:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.697331:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.697374:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac80d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8109 expref: 16 pid: 11036 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.697389:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.697395:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.697405:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.697414:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.697454:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.697469:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660226048:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845026.697553:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.701821:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660226048:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845026.701861:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845026.701977:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641492352:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4710us (4941us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845026.701989:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 309 00000100:00100000:0.0:1788845026.702854:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.703007:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641492480 00000100:00100000:2.0:1788845026.703015:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641492480 00000100:00100000:2.0:1788845026.703085:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 508 00000100:00100000:2.0:1788845026.703096:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641492480:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845026.703349:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:2.0:1788845026.703353:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845026.703356:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845026.703359:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845026.703360:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:2.0:1788845026.703421:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641492480:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 326us (573us total) trans 4294967640 rc 0/0 00000100:00100000:2.0:1788845026.703429:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 508 00000100:00100000:2.0:1788845026.713023:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.722109:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641492608 00000100:00100000:3.0:1788845026.722133:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641492608 00000100:00100000:3.0:1788845026.722209:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 848 00000100:00100000:3.0:1788845026.722219:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641492608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.722240:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.722267:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a815d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.722299:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.722324:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.722361:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.722379:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 12 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.722393:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660226176:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845026.722470:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.724383:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660226176:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845026.724415:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.724434:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.724437:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845026.724441:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.724451:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960c00, rc 0) 00000100:00100000:3.0:1788845026.724505:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641492608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2287us (11493us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845026.724513:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 848 00000100:00100000:0.0:1788845026.733046:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.736243:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641492736 00000100:00100000:3.0:1788845026.736275:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641492736 00010000:00010000:3.0:1788845026.736323:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 13 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.736375:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 310 00000100:00100000:3.0:1788845026.736385:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641492736:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845026.736402:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845026.736429:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 13 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.736446:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 13 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.736466:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac861 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8156 expref: 13 pid: 5839 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.736482:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.736489:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.736500:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.736509:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.736550:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.736575:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660226304:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845026.736668:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845026.740932:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.741029:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641492864 00000100:00100000:1.0:1788845026.741038:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641492864 00000100:00100000:1.0:1788845026.741122:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 509 00000100:00100000:1.0:1788845026.741132:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641492864:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845026.741358:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:1.0:1788845026.741362:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845026 -> 1788845026 00000004:00010000:1.0:1788845026.741365:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:1.0:1788845026.741367:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845026 -> 1788845026 00000004:00010000:1.0:1788845026.741369:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:1.0:1788845026.741441:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641492864:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 310us (519us total) trans 4294967641 rc 0/0 00000100:00100000:1.0:1788845026.741449:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 509 00000100:00100000:3.0:1788845026.742075:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660226304:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845026.742098:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845026.742134:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641492736:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5751us (9100us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845026.742143:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 310 00000100:00100000:2.0:1788845026.746548:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.746647:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641492992 00000100:00100000:1.0:1788845026.746659:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641492992 00000100:00100000:1.0:1788845026.746720:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 849 00000100:00100000:1.0:1788845026.746730:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641492992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.746756:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.746778:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac86f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8164 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.746879:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.746934:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.746947:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845026.746987:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1439494 00000004:00200000:1.0:1788845026.747006:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845026.747026:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.747050:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac86f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.747068:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.747077:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.747079:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845026.747082:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2200, rc 0) 00000100:00100000:1.0:1788845026.747145:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641492992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 416us (607us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845026.747152:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 849 00000100:00100000:0.0:1788845026.753602:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.753691:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641493120 00000100:00100000:2.0:1788845026.753702:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641493120 00000100:00100000:2.0:1788845026.753756:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 557 00000100:00100000:2.0:1788845026.753764:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641493120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.753797:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.753813:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a816b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.753847:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:2.0:1788845026.753852:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:2.0:1788845026.753862:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.753879:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.754196:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1000, rc 0) 00000100:00100000:2.0:1788845026.754262:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641493120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 498us (668us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.754270:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 557 00000100:00100000:0.0:1788845026.757504:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.757586:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641493248 00000100:00100000:1.0:1788845026.757596:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641493248 00000100:00100000:1.0:1788845026.757658:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 558 00000100:00100000:1.0:1788845026.757669:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641493248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.757689:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.757709:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8172 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.757741:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.757751:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.757773:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:1.0:1788845026.757777:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:1.0:1788845026.757782:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1871872->1896447] (req 1871872->1896447) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.757818:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.757852:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 9 pid: 7983 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.757869:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660226432:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845026.757997:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.758008:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 9 pid: 6684 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.758018:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660226560:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845026.758042:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.764544:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660226560:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845026.769100:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.769170:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.769288:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641493376 00000100:00100000:2.0:1788845026.769300:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641493376 00000100:00100000:3.0:1788845026.769338:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641493504 00000100:00100000:3.0:1788845026.769344:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641493504 00010000:00010000:2.0:1788845026.769349:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.769369:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 10 pid: 7983 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.769398:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 311 00000100:00100000:2.0:1788845026.769410:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641493376:12345-192.168.201.29@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1788845026.769415:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 312 00000100:00100000:3.0:1788845026.769427:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641493504:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845026.769430:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845026.769439:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845026.769453:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.769457:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 10 pid: 7983 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.769493:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 10 pid: 7983 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.769554:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660226432:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845026.770378:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.770410:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac84c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8141 expref: 10 pid: 6684 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.770432:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845026.770487:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641493376:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 1079us (1397us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.770495:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 311 00010000:00010000:3.0:1788845026.770822:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac87d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a816b expref: 10 pid: 7983 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.770864:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788845026.770868:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788845026.770877:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.770915:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.770965:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.770980:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660226688:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845026.771039:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00010000:00010000:1.0:1788845026.773002:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.773026:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3000, rc 0) 00000100:00100000:2.0:1788845026.774947:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660226688:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845026.774982:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845026.775033:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641493504:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5608us (5861us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.775044:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 312 00000100:00100000:0.0:1788845026.783111:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641493248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 25429us (25603us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845026.783126:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 558 00000100:00100000:3.0:1788845026.791445:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.791525:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.791677:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641493760 00000100:00100000:2.0:1788845026.791688:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641493760 00000100:00100000:2.0:1788845026.791749:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 559 00000100:00100000:2.0:1788845026.791761:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641493760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.791781:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.791799:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac88b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8180 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.791876:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035685180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660226816:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845026.792010:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.792918:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641493632 00000100:00100000:1.0:1788845026.792928:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641493632 00000100:00100000:1.0:1788845026.792991:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 850 00000100:00100000:1.0:1788845026.793000:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641493632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.793022:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.793041:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2400/0xb2187301287ac892 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8179 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.793079:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.793111:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660226944:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.793163:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845026.799753:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1439494 -> 1894431 00000100:00100000:2.0:1788845026.799783:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035685180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660226816:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845026.799821:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac88b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8180 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.799858:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac88b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8180 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.799872:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845026.800002:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641493760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8242us (8475us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845026.800011:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 559 00000100:00100000:1.0:1788845026.802864:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660226944:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845026.802931:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845026.802959:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2400/0xb2187301287ac892 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8179 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.802984:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.802987:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845026.802995:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2400/0xb2187301287ac892 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8179 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.803031:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2400, rc 0) 00000100:00100000:1.0:1788845026.803129:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641493632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10129us (11696us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.803138:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 850 00000100:00100000:0.0:1788845026.805127:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.805169:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641493888 00000100:00100000:1.0:1788845026.805180:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641493888 00000100:00100000:1.0:1788845026.805243:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 851 00000100:00100000:1.0:1788845026.805253:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641493888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.805273:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.805294:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac899 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8187 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.805343:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.805392:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660227072:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.805994:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845026.806398:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.806553:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641494016 00000100:00100000:3.0:1788845026.806559:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641494016 00000100:00100000:3.0:1788845026.806601:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 560 00000100:00100000:3.0:1788845026.806607:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641494016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.806621:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.806635:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bbea8b7d/0xb2187301287ac8a0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a818e expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.806672:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660227200:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845026.806717:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.812862:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660227200:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845026.812936:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bbea8b7d/0xb2187301287ac8a0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a818e expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.812969:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bbea8b7d/0xb2187301287ac8a0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a818e expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.812981:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960200, rc 0) 00000100:00100000:3.0:1788845026.813036:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641494016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6428us (6639us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845026.813045:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 560 00000100:00100000:1.0:1788845026.813679:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660227072:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845026.813700:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845026.813714:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac899 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8187 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.813729:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.813731:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845026.813735:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3800/0xb2187301287ac899 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8187 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.813762:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3800, rc 0) 00000100:00100000:1.0:1788845026.813793:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641493888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8542us (8673us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.813800:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 851 00000100:00100000:0.0:1788845026.836558:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.836606:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.836692:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641494144 00000100:00100000:3.0:1788845026.836702:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641494144 00000100:00100000:3.0:1788845026.836763:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 852 00000100:00100000:3.0:1788845026.836775:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641494144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.836795:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.836817:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac8a7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8195 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.836868:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.836935:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660227328:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845026.837015:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845026.837098:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641494272 00000100:00100000:2.0:1788845026.837103:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641494272 00000100:00100000:2.0:1788845026.837149:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 561 00000100:00100000:2.0:1788845026.837157:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641494272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.837166:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.837186:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287ac8ae lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a819c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.837225:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660227456:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845026.837252:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.840935:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660227328:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845026.840998:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845026.841019:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac8a7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8195 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.841041:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.841044:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845026.841049:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac8a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8195 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.841064:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960200, rc 0) 00000100:00100000:2.0:1788845026.841174:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660227456:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845026.841217:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287ac8ae lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a819c expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.841237:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287ac8ae lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a819c expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.841247:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1800, rc 0) 00000100:00100000:3.0:1788845026.841348:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641494144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4573us (4799us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845026.841357:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 852 00000100:00100000:2.0:1788845026.850192:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641494272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13019us (13567us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845026.850207:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 561 00000100:00100000:0.0:1788845026.854302:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.854377:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.854451:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641494528 00000100:00100000:2.0:1788845026.854463:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641494528 00000100:00100000:2.0:1788845026.854538:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 562 00000100:00100000:2.0:1788845026.854548:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641494528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.854567:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.854587:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287ac8b5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81aa expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.854677:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660227584:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845026.854769:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.856050:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641494400 00000100:00100000:3.0:1788845026.856060:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641494400 00000100:00100000:3.0:1788845026.856418:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 853 00000100:00100000:3.0:1788845026.856428:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641494400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.856450:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.856481:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac8bc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81a3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.856521:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.856564:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660227712:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845026.856654:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.858683:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660227712:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845026.858710:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845026.858728:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac8bc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81a3 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.858751:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.858754:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845026.858759:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960200/0xb2187301287ac8bc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81a3 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.858774:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960200, rc 0) 00000100:00100000:3.0:1788845026.858822:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641494400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2395us (4713us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845026.858842:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 853 00000100:00100000:2.0:1788845026.858979:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660227584:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845026.859002:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287ac8b5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81aa expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.859020:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287ac8b5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81aa expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845026.859050:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:2.0:1788845026.859072:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641494528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4526us (4695us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845026.859079:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 562 00000100:00100000:2.0:1788845026.862705:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.862753:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.862820:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641494784 00000100:00100000:3.0:1788845026.862830:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641494784 00000100:00100000:3.0:1788845026.862921:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 563 00000100:00100000:3.0:1788845026.862932:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641494784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.862965:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.862984:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006dfbc169/0xb2187301287ac8c3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81b8 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.863070:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660227840:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845026.863154:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.863451:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641494656 00000100:00100000:1.0:1788845026.863456:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641494656 00000100:00100000:1.0:1788845026.863496:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 854 00000100:00100000:1.0:1788845026.863503:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a005694c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641494656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.863514:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.863527:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3600/0xb2187301287ac8ca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81b1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.863554:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.863578:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660227968:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.863623:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.866813:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660227968:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845026.866843:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845026.866863:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3600/0xb2187301287ac8ca lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81b1 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.866926:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.866929:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845026.866934:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3600/0xb2187301287ac8ca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81b1 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.866976:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3600, rc 0) 00000100:00100000:1.0:1788845026.867032:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a005694c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641494656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3528us (4338us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.867040:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 854 00000100:00100000:3.0:1788845026.869731:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660227840:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845026.869770:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006dfbc169/0xb2187301287ac8c3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81b8 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.869821:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006dfbc169/0xb2187301287ac8c3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81b8 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.869835:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961a00, rc 0) 00000100:00100000:3.0:1788845026.869918:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641494784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6986us (7163us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845026.869927:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 563 00000100:00100000:2.0:1788845026.873973:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.874012:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.874148:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641494912 00000100:00100000:1.0:1788845026.874158:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641494912 00000100:00100000:1.0:1788845026.874231:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 855 00000100:00100000:1.0:1788845026.874243:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641494912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845026.874266:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845026.874289:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac8d1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81bf expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.874331:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845026.874379:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660228096:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845026.874460:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.874752:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641495040 00000100:00100000:0.0:1788845026.874758:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641495040 00000100:00100000:0.0:1788845026.874802:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 564 00000100:00100000:0.0:1788845026.874810:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641495040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845026.874820:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845026.874832:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000453aa122/0xb2187301287ac8d8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81c6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845026.874869:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660228224:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845026.874929:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845026.879381:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660228096:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845026.879415:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845026.879436:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac8d1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81bf expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.879457:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845026.879460:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845026.879475:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac8d1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81bf expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.879507:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2c00, rc 0) 00000100:00100000:1.0:1788845026.880073:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641494912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5830us (6057us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.880082:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 855 00000100:00100000:1.0:1788845026.882067:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660228224:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845026.882122:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000453aa122/0xb2187301287ac8d8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81c6 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.882150:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000453aa122/0xb2187301287ac8d8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81c6 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845026.882163:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2200, rc 0) 00000100:00100000:1.0:1788845026.882236:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641495040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7426us (8271us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845026.882245:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 564 00000100:00100000:2.0:1788845026.888497:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845026.888735:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641495168 00000100:00100000:3.0:1788845026.888746:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641495168 00000100:00100000:3.0:1788845026.888809:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 856 00000100:00100000:3.0:1788845026.888820:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641495168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845026.889019:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845026.889045:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287ac8df lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81cd expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.889080:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.889092:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287ac8df lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81cd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.889312:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.889333:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 17 pid: 11036 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.889360:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660228352:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845026.889435:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a002d1bb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845026.895672:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845026.895763:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641495296 00000100:00100000:0.0:1788845026.895774:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641495296 00010000:00010000:0.0:1788845026.895818:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 18 pid: 11036 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845026.896043:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 313 00000100:00100000:0.0:1788845026.896054:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008914c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641495296:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845026.896069:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845026.896098:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 18 pid: 11036 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.896127:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 18 pid: 11036 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.896154:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845026.896160:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287ac8df lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81cd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.896180:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287ac8df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81cd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.896190:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287ac8df lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81cd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.896229:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287ac8df lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81cd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845026.896245:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660228480:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845026.896484:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960c00/0xb2187301287ac868 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a815d expref: 18 pid: 11036 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845026.896501:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660228352:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845026.896528:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287ac8df lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81cd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845026.896538:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845026.896541:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845026.896543:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961a00, rc 0) 00000100:00100000:0.0:1788845026.896743:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845026.897067:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641495168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8247us (8580us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845026.897077:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 856 00010000:00010000:1.0:1788845026.901437:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845026.901808:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.901914:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641495424 00000100:00100000:2.0:1788845026.901923:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641495424 00000100:00100000:2.0:1788845026.901981:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 565 00000100:00100000:2.0:1788845026.901991:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641495424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845026.902009:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845026.902025:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287ac8e6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81d4 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845026.902084:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660228608:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845026.902166:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845026.905193:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660228608:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845026.905234:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287ac8e6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81d4 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.905260:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287ac8e6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81d4 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845026.905272:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:0.0:1788845026.905338:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641495424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3348us (3536us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845026.905347:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 565 00000100:00100000:1.0:1788845026.908699:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660228480:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845026.908728:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845026.912860:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.913089:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641495552 00000100:00100000:2.0:1788845026.913101:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641495552 00002000:00100000:2.0:1788845026.913179:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383100 x1875739641495552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 504/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.913192:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383100 with x1875739641495552 00000100:00100000:2.0:1788845026.913252:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 510 00000100:00100000:2.0:1788845026.913263:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641495552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845026.916754:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008914c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641495296:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 20689us (21080us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845026.916773:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 313 00000100:00100000:0.0:1788845026.933274:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641495552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 19999us (20412us total) trans 0 rc 1007616/1007616 00000100:00100000:0.0:1788845026.933290:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 510 00002000:00100000:0.0:1788845026.933304:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383100 x1875739641495552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 504/440 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 1007616/1007616 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845026.933315:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383100 with x1875739641495552 00000100:00100000:2.0:1788845026.950564:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.950664:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641495680 00000100:00100000:1.0:1788845026.950674:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641495680 00002000:00100000:1.0:1788845026.950722:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687480 x1875739641495680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.950733:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687480 with x1875739641495680 00000100:00100000:1.0:1788845026.950805:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 511 00000100:00100000:1.0:1788845026.950829:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641495680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.951181:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641495680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 352us (627us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845026.951189:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 511 00002000:00100000:1.0:1788845026.951197:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687480 x1875739641495680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/440 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.951206:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687480 with x1875739641495680 00000100:00100000:3.0:1788845026.955860:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.956051:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641495808 00000100:00100000:1.0:1788845026.956061:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641495808 00002000:00100000:1.0:1788845026.956111:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02381f80 x1875739641495808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.956123:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02381f80 with x1875739641495808 00000100:00100000:1.0:1788845026.956149:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 512 00000100:00100000:1.0:1788845026.956158:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641495808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.956444:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641495808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 285us (593us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845026.956452:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 512 00002000:00100000:1.0:1788845026.956459:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02381f80 x1875739641495808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/440 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.956466:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02381f80 with x1875739641495808 00000100:00100000:3.0:1788845026.966090:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845026.966305:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641495936 00000100:00100000:2.0:1788845026.966317:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641495936 00002000:00100000:2.0:1788845026.966371:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380380 x1875739641495936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.966384:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380380 with x1875739641495936 00000100:00100000:2.0:1788845026.966430:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 513 00000100:00100000:2.0:1788845026.966440:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641495936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845026.966816:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641495936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 376us (734us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845026.966834:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 513 00002000:00100000:2.0:1788845026.966842:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380380 x1875739641495936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/440 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845026.966850:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380380 with x1875739641495936 00000100:00100000:2.0:1788845026.975311:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845026.975680:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641496064 00000100:00100000:1.0:1788845026.975691:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641496064 00002000:00100000:1.0:1788845026.975744:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687b80 x1875739641496064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/0 e 0 to 0 dl 1788845037 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.975756:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687b80 with x1875739641496064 00000100:00100000:1.0:1788845026.975814:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 514 00000100:00100000:1.0:1788845026.975926:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641496064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845026.976942:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641496064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1115us (1642us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845026.976961:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 514 00002000:00100000:1.0:1788845026.976971:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687b80 x1875739641496064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:132/0 lens 488/440 e 0 to 0 dl 1788845037 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845026.976981:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687b80 with x1875739641496064 00000100:00100000:2.0:1788845027.001746:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.001833:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641496192 00000100:00100000:1.0:1788845027.001843:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641496192 00000100:00100000:1.0:1788845027.001935:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 566 00000100:00100000:1.0:1788845027.001956:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641496192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.001975:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.001994:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81db expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.002027:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.002038:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(5 12) 00010000:00010000:1.0:1788845027.002042:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:1.0:1788845027.002048:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1081343] (req 1048576->1081343) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.002080:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.002127:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 7 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.002141:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660228736:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845027.002227:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00080000:2.0:1788845027.002350:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845027.008285:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660228736:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845027.008327:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1081343] (req 1048576->1081343) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.008348:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3600, rc 0) 00000100:00100000:1.0:1788845027.008408:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641496192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6464us (6672us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845027.008415:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 566 00000100:00100000:3.0:1788845027.011194:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.011489:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641496320 00000100:00100000:1.0:1788845027.011499:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641496320 00002000:00100000:1.0:1788845027.011611:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1871872->1894430) req@ffff949f02383800 x1875739641496320/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.011624:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02383800 with x1875739641496320 ext(1871872->1894430) 00010000:00010000:1.0:1788845027.011634:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 8 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.011657:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 8 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845027.011686:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f02383800 00000100:00100000:1.0:1788845027.011717:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 13 00000100:00100000:1.0:1788845027.011728:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641496320:12345-192.168.201.29@tcp:4:fsx.0 00000100:00080000:2.0:1788845027.020857:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845027.022056:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641496320:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 10321us (10864us total) trans 4294967405 rc 0/0 00000100:00100000:1.0:1788845027.022069:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 13 00002000:00100000:1.0:1788845027.022082:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1871872->1894430) req@ffff949f02383800 x1875739641496320/t4294967405(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/448 e 0 to 0 dl 1788845038 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.022093:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02383800 with x1875739641496320 ext(1871872->1894430) 00010000:00010000:1.0:1788845027.022104:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 10 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.022137:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 10 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845027.022147:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f02383800 00000100:00100000:1.0:1788845027.026210:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035686a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660228864:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788845027.026291:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845027.026450:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660228864 00000100:00100000:0.0:1788845027.026459:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660228864 00000100:00100000:0.0:1788845027.026614:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 78 00000100:00100000:0.0:1788845027.026626:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660228864:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845027.026662:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00080000:2.0:1788845027.026784:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845027.027924:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660228864:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1292us (1624us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845027.027937:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 78 00000100:00100000:3.0:1788845027.028243:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.028469:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641496448 00000100:00100000:0.0:1788845027.028478:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641496448 00010000:00010000:0.0:1788845027.028512:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 10 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.028551:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 314 00000100:00100000:0.0:1788845027.028560:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641496448:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845027.028578:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845027.028605:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 10 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.028626:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 10 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.028654:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c60edf2/0xb2187301287ac884 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1871872->1896447) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8172 expref: 10 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.028672:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:0.0:1788845027.028676:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:0.0:1788845027.028683:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.028690:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.028736:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.028747:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660228992:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845027.028816:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.030127:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035686a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660228864:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845027.038836:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.038904:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.038933:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660228992:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845027.038967:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845027.038997:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641496704 00000100:00100000:1.0:1788845027.039004:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641496704 00000100:00100000:0.0:1788845027.039032:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641496448:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10473us (10791us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845027.039042:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 314 00002000:00100000:1.0:1788845027.039067:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686a00 x1875739641496704/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00080000:0.0:1788845027.039093:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00100000:1.0:1788845027.039097:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686a00 with x1875739641496704 00000100:00100000:1.0:1788845027.039121:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 515 00000100:00100000:1.0:1788845027.039131:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641496704:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845027.039279:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641496576 00000100:00100000:3.0:1788845027.039284:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641496576 00002000:00100000:3.0:1788845027.039307:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1896447) req@ffff94a035685500 x1875739641496576/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845027.039317:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035685500 with x1875739641496576 ext(1048576->1896447) 00002000:00010000:3.0:1788845027.039328:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035685500 00000100:00100000:3.0:1788845027.039350:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 330 00000100:00100000:3.0:1788845027.039357:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641496576:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.051835:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641496704:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12678us (12925us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845027.051856:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 515 00002000:00100000:1.0:1788845027.051873:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686a00 x1875739641496704/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/440 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.051902:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686a00 with x1875739641496704 00000100:00100000:2.0:1788845027.074781:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641496576:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 35413us (35944us total) trans 0 rc 847872/847872 00000100:00100000:2.0:1788845027.074796:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 330 00002000:00100000:2.0:1788845027.074831:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1896447) req@ffff94a035685500 x1875739641496576/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/440 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 847872/847872 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.074842:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035685500 with x1875739641496576 ext(1048576->1896447) 00002000:00010000:2.0:1788845027.074848:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035685500 00000100:00100000:1.0:1788845027.082328:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.082452:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.082650:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641496960 00000100:00100000:2.0:1788845027.082662:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641496960 00000100:00100000:2.0:1788845027.082739:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 567 00000100:00100000:2.0:1788845027.082751:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641496960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.082777:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.083055:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81e9 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.083087:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:2.0:1788845027.083092:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:2.0:1788845027.083104:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.083124:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.083141:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845027.083262:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2368:x1875739641496960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 512us (808us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845027.083269:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 567 00000100:00100000:3.0:1788845027.083352:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641496832 00000100:00100000:3.0:1788845027.083358:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641496832 00000100:00100000:3.0:1788845027.083423:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 857 00000100:00100000:3.0:1788845027.083433:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641496832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845027.083446:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845027.083461:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81e2 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.083502:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845027.083514:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845027.083520:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.083532:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845027.083535:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845027.083539:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:3.0:1788845027.083563:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641496832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 132us (1246us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845027.083569:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 857 00000100:00080000:2.0:1788845027.083628:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845027.093614:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.095196:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641497088 00000100:00100000:1.0:1788845027.095210:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641497088 00002000:00100000:1.0:1788845027.095271:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1896447) req@ffff94a035684000 x1875739641497088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.095283:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035684000 with x1875739641497088 ext(1048576->1896447) 00002000:00010000:1.0:1788845027.095290:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035684000 00000100:00100000:1.0:1788845027.095337:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 331 00000100:00100000:1.0:1788845027.095349:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641497088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845027.096581:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.096619:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.096645:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641497344 00000100:00100000:3.0:1788845027.096660:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641497344 00000100:00100000:3.0:1788845027.096704:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 279 00000100:00100000:3.0:1788845027.096714:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641497344:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845027.096779:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641497344:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 66us (159us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845027.096786:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 279 00000100:00100000:2.0:1788845027.097437:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641497216 00000100:00100000:2.0:1788845027.097444:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641497216 00002000:00100000:2.0:1788845027.097496:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02382d80 x1875739641497216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.097507:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02382d80 with x1875739641497216 00000100:00100000:2.0:1788845027.097536:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 516 00000100:00100000:2.0:1788845027.097543:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641497216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.108302:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641497216:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 10747us (11716us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845027.108317:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 516 00002000:00100000:2.0:1788845027.108332:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02382d80 x1875739641497216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/440 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.108342:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02382d80 with x1875739641497216 00000100:00100000:1.0:1788845027.113493:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641497088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 18125us (19867us total) trans 0 rc 847872/847872 00000100:00100000:1.0:1788845027.113511:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 331 00002000:00100000:1.0:1788845027.113525:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1896447) req@ffff94a035684000 x1875739641497088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/440 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 847872/847872 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.113549:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035684000 with x1875739641497088 ext(1048576->1896447) 00002000:00010000:1.0:1788845027.113556:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035684000 00000100:00100000:0.0:1788845027.148760:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.148917:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641497472 00000100:00100000:2.0:1788845027.148931:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641497472 00002000:00100000:2.0:1788845027.148998:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01dbb480 x1875739641497472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.149011:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01dbb480 with x1875739641497472 00000100:00100000:2.0:1788845027.149054:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 517 00000100:00100000:2.0:1788845027.149064:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641497472:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.149362:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641497472:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 299us (616us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.149371:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 517 00002000:00100000:2.0:1788845027.149378:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01dbb480 x1875739641497472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.149405:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01dbb480 with x1875739641497472 00000100:00100000:3.0:1788845027.153273:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.153536:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641497600 00000100:00100000:1.0:1788845027.153547:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641497600 00002000:00100000:1.0:1788845027.153602:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380a80 x1875739641497600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.153613:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380a80 with x1875739641497600 00000100:00100000:1.0:1788845027.153656:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 518 00000100:00100000:1.0:1788845027.153667:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641497600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.154026:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641497600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 360us (765us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.154033:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 518 00002000:00100000:1.0:1788845027.154040:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380a80 x1875739641497600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.154047:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380a80 with x1875739641497600 00000100:00100000:3.0:1788845027.166924:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.167078:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641497728 00000100:00100000:2.0:1788845027.167089:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641497728 00002000:00100000:2.0:1788845027.167138:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02382a00 x1875739641497728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.167150:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02382a00 with x1875739641497728 00000100:00100000:2.0:1788845027.167200:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 519 00000100:00100000:2.0:1788845027.167211:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641497728:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.167481:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641497728:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 272us (604us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.167489:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 519 00002000:00100000:2.0:1788845027.167496:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02382a00 x1875739641497728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.167504:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02382a00 with x1875739641497728 00000100:00100000:0.0:1788845027.177601:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.178938:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641497856 00000100:00100000:1.0:1788845027.178961:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641497856 00002000:00100000:1.0:1788845027.179016:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01dba680 x1875739641497856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.179027:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01dba680 with x1875739641497856 00000100:00100000:1.0:1788845027.179071:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 520 00000100:00100000:1.0:1788845027.179081:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641497856:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.180275:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641497856:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1190us (2677us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.180299:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 520 00002000:00100000:1.0:1788845027.180309:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01dba680 x1875739641497856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.180319:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01dba680 with x1875739641497856 00000100:00100000:3.0:1788845027.195296:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.195409:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641497984 00000100:00100000:2.0:1788845027.195419:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641497984 00002000:00100000:2.0:1788845027.195471:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380000 x1875739641497984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.195607:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380000 with x1875739641497984 00000100:00100000:2.0:1788845027.195651:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 521 00000100:00100000:2.0:1788845027.195662:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641497984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.195956:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641497984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 285us (659us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.195982:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 521 00002000:00100000:2.0:1788845027.195990:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380000 x1875739641497984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.195997:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380000 with x1875739641497984 00000100:00100000:3.0:1788845027.199014:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.199294:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641498112 00000100:00100000:1.0:1788845027.199306:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641498112 00002000:00100000:1.0:1788845027.199864:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02381180 x1875739641498112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.199876:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02381180 with x1875739641498112 00000100:00100000:1.0:1788845027.200066:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 522 00000100:00100000:1.0:1788845027.200077:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641498112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.200390:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641498112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 314us (1386us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.200399:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 522 00002000:00100000:1.0:1788845027.200406:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02381180 x1875739641498112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.200414:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02381180 with x1875739641498112 00000100:00100000:0.0:1788845027.225098:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.225539:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641498240 00000100:00100000:1.0:1788845027.225550:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641498240 00000100:00100000:1.0:1788845027.225615:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 858 00000100:00100000:1.0:1788845027.225626:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641498240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.225647:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.225651:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788845027.225693:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287ac8df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81cd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.225729:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.225733:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845027.225747:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81f0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.225771:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.225780:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.225812:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.225830:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 17 pid: 5839 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.225843:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660229120:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845027.225988:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845027.231124:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.231237:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641498368 00000100:00100000:2.0:1788845027.231248:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641498368 00010000:00010000:2.0:1788845027.231301:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 18 pid: 5839 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.231353:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 315 00000100:00100000:2.0:1788845027.231364:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641498368:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845027.231380:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845027.231408:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 18 pid: 5839 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.231426:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 18 pid: 5839 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.231447:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.231454:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.231476:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.231486:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.231539:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.231552:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660229248:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845027.231638:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00010000:00010000:1.0:1788845027.233049:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ac8fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e2 expref: 18 pid: 5839 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.233069:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660229120:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845027.233096:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.233140:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.233144:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.233146:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2e00, rc 0) 00000100:00100000:1.0:1788845027.233198:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641498240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7574us (8110us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845027.233206:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 858 00000100:00100000:2.0:1788845027.244079:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660229248:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845027.244108:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845027.244178:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641498368:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 12816us (13064us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845027.244190:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 315 00000100:00100000:3.0:1788845027.255537:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.255755:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641498496 00000100:00100000:2.0:1788845027.255765:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641498496 00002000:00100000:2.0:1788845027.255815:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f825f80 x1875739641498496/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.255835:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f825f80 with x1875739641498496 00000100:00100000:2.0:1788845027.255873:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 523 00000100:00100000:2.0:1788845027.255919:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641498496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.256198:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641498496:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 316us (669us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.256206:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 523 00002000:00100000:2.0:1788845027.256214:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f825f80 x1875739641498496/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.256222:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f825f80 with x1875739641498496 00000100:00100000:3.0:1788845027.260285:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.260403:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641498624 00000100:00100000:2.0:1788845027.260414:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641498624 00002000:00100000:2.0:1788845027.260524:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826300 x1875739641498624/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.260535:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826300 with x1875739641498624 00000100:00100000:2.0:1788845027.260574:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 524 00000100:00100000:2.0:1788845027.260584:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641498624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.261346:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641498624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 758us (1463us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.261355:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 524 00002000:00100000:2.0:1788845027.261364:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826300 x1875739641498624/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.261374:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826300 with x1875739641498624 00000100:00100000:3.0:1788845027.264206:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.264285:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641498752 00000100:00100000:1.0:1788845027.264304:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641498752 00002000:00100000:1.0:1788845027.264352:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f825180 x1875739641498752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.264363:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f825180 with x1875739641498752 00000100:00100000:1.0:1788845027.264409:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 525 00000100:00100000:1.0:1788845027.264418:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641498752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.264699:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641498752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (502us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.264706:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 525 00002000:00100000:1.0:1788845027.264713:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f825180 x1875739641498752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.264720:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f825180 with x1875739641498752 00000100:00100000:3.0:1788845027.268208:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.268314:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641498880 00000100:00100000:1.0:1788845027.268335:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641498880 00002000:00100000:1.0:1788845027.268387:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824380 x1875739641498880/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.268398:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824380 with x1875739641498880 00000100:00100000:1.0:1788845027.268453:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 526 00000100:00100000:1.0:1788845027.268463:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641498880:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.268732:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641498880:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 269us (532us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.268741:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 526 00002000:00100000:1.0:1788845027.268747:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824380 x1875739641498880/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.268756:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824380 with x1875739641498880 00000100:00100000:0.0:1788845027.278191:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.278773:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641499008 00000100:00100000:1.0:1788845027.278782:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641499008 00002000:00100000:1.0:1788845027.278904:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db8700 x1875739641499008/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.278913:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db8700 with x1875739641499008 00000100:00100000:1.0:1788845027.278951:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 527 00000100:00100000:1.0:1788845027.278961:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641499008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.279846:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641499008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 880us (1660us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.279856:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 527 00002000:00100000:1.0:1788845027.279865:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db8700 x1875739641499008/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.279875:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db8700 with x1875739641499008 00000100:00100000:3.0:1788845027.287284:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.287523:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641499136 00000100:00100000:1.0:1788845027.287534:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641499136 00002000:00100000:1.0:1788845027.287588:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b7100 x1875739641499136/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.287616:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7100 with x1875739641499136 00000100:00100000:1.0:1788845027.287650:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 528 00000100:00100000:1.0:1788845027.287659:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641499136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.287935:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641499136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (661us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.287951:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 528 00002000:00100000:1.0:1788845027.287959:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b7100 x1875739641499136/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.287967:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7100 with x1875739641499136 00000100:00100000:2.0:1788845027.290989:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.291133:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641499264 00000100:00100000:0.0:1788845027.291144:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641499264 00002000:00100000:0.0:1788845027.291210:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687800 x1875739641499264/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845027.291221:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687800 with x1875739641499264 00000100:00100000:0.0:1788845027.291301:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 529 00000100:00100000:0.0:1788845027.291321:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641499264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845027.291592:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641499264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (616us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845027.291614:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 529 00002000:00100000:0.0:1788845027.291621:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687800 x1875739641499264/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845027.291628:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687800 with x1875739641499264 00000100:00100000:0.0:1788845027.296100:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.296665:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641499392 00000100:00100000:2.0:1788845027.296676:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641499392 00002000:00100000:2.0:1788845027.296726:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01dbad80 x1875739641499392/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.296737:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01dbad80 with x1875739641499392 00000100:00100000:2.0:1788845027.296777:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 530 00000100:00100000:2.0:1788845027.296787:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641499392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.297837:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641499392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1046us (1742us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.297847:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 530 00002000:00100000:2.0:1788845027.297856:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01dbad80 x1875739641499392/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.297878:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01dbad80 with x1875739641499392 00000100:00100000:3.0:1788845027.303031:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.303293:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641499520 00000100:00100000:2.0:1788845027.303305:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641499520 00002000:00100000:2.0:1788845027.303359:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5180 x1875739641499520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.303370:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5180 with x1875739641499520 00000100:00100000:2.0:1788845027.303409:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 531 00000100:00100000:2.0:1788845027.303419:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641499520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.303679:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641499520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (656us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.303686:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 531 00002000:00100000:2.0:1788845027.303693:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5180 x1875739641499520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.303700:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5180 with x1875739641499520 00000100:00100000:3.0:1788845027.309546:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.309852:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641499648 00000100:00100000:1.0:1788845027.309864:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641499648 00002000:00100000:1.0:1788845027.309965:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b6300 x1875739641499648/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.309976:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6300 with x1875739641499648 00000100:00100000:1.0:1788845027.310027:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 532 00000100:00100000:1.0:1788845027.310038:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641499648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.310474:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641499648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 435us (934us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.310483:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 532 00002000:00100000:1.0:1788845027.310490:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b6300 x1875739641499648/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.310498:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6300 with x1875739641499648 00000100:00100000:1.0:1788845027.320475:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.320654:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641499776 00000100:00100000:2.0:1788845027.320666:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641499776 00002000:00100000:2.0:1788845027.320728:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03568aa00 x1875739641499776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.320739:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03568aa00 with x1875739641499776 00000100:00100000:2.0:1788845027.320779:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 533 00000100:00100000:2.0:1788845027.320788:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641499776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.321999:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641499776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1204us (1540us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.322022:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 533 00002000:00100000:2.0:1788845027.322031:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03568aa00 x1875739641499776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.322040:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03568aa00 with x1875739641499776 00000100:00100000:3.0:1788845027.326799:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.326921:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641499904 00000100:00100000:1.0:1788845027.326932:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641499904 00002000:00100000:1.0:1788845027.326991:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4380 x1875739641499904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.327002:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4380 with x1875739641499904 00000100:00100000:1.0:1788845027.327042:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 534 00000100:00100000:1.0:1788845027.327057:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641499904:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.327357:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641499904:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 305us (566us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.327365:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 534 00002000:00100000:1.0:1788845027.327373:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4380 x1875739641499904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.327381:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4380 with x1875739641499904 00000100:00100000:3.0:1788845027.331283:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.331450:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641500032 00000100:00100000:0.0:1788845027.331471:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641500032 00002000:00100000:0.0:1788845027.331520:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4000 x1875739641500032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845027.331530:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4000 with x1875739641500032 00000100:00100000:0.0:1788845027.331572:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 535 00000100:00100000:0.0:1788845027.331582:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641500032:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845027.331841:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641500032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 258us (565us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845027.331848:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 535 00002000:00100000:0.0:1788845027.331856:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4000 x1875739641500032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845027.331863:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4000 with x1875739641500032 00000100:00100000:2.0:1788845027.335683:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.335787:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641500160 00000100:00100000:1.0:1788845027.335797:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641500160 00002000:00100000:1.0:1788845027.335854:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01ceca80 x1875739641500160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.335865:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01ceca80 with x1875739641500160 00000100:00100000:1.0:1788845027.335934:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 536 00000100:00100000:1.0:1788845027.335954:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641500160:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.336225:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641500160:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (551us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.336234:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 536 00002000:00100000:1.0:1788845027.336241:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01ceca80 x1875739641500160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.336269:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01ceca80 with x1875739641500160 00000100:00100000:1.0:1788845027.340638:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.341098:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641500288 00000100:00100000:0.0:1788845027.341108:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641500288 00002000:00100000:1.0:1788845027.341312:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035688700 x1875739641500288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.341325:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035688700 with x1875739641500288 00000100:00100000:1.0:1788845027.341412:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 537 00000100:00100000:1.0:1788845027.341423:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641500288:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.341704:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641500288:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (1075us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.341713:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 537 00002000:00100000:1.0:1788845027.341720:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035688700 x1875739641500288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.341728:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035688700 with x1875739641500288 00000100:00100000:2.0:1788845027.345811:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.345992:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641500416 00000100:00100000:1.0:1788845027.346001:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641500416 00002000:00100000:1.0:1788845027.346052:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01ceea00 x1875739641500416/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.346063:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01ceea00 with x1875739641500416 00000100:00100000:1.0:1788845027.346104:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 538 00000100:00100000:1.0:1788845027.346115:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641500416:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845027.352192:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:1.0:1788845027.352204:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845026 -> 1788845027 00000004:00010000:1.0:1788845027.352207:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:1.0:1788845027.352219:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845026 -> 1788845027 00000004:00010000:1.0:1788845027.352221:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:1.0:1788845027.352317:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641500416:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 6199us (6511us total) trans 4294967642 rc 0/0 00000100:00100000:1.0:1788845027.352329:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 538 00002000:00100000:1.0:1788845027.352342:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01ceea00 x1875739641500416/t4294967642(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/448 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.352352:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01ceea00 with x1875739641500416 00000100:00100000:3.0:1788845027.360285:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.360451:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641500544 00000100:00100000:0.0:1788845027.360470:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641500544 00000100:00100000:0.0:1788845027.360533:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 859 00000100:00100000:0.0:1788845027.360543:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641500544:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845027.422161:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967638 is committed 00000001:00080000:3.0:1788845027.422193:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967642 is committed 00000001:00080000:3.0:1788845027.422204:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967641 is committed 00000100:00100000:2.0:1788845027.422339:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641500544:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 61791us (62060us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845027.422353:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 859 00000100:00100000:2.0:1788845027.428145:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.428482:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641500672 00000100:00100000:3.0:1788845027.428495:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641500672 00000100:00100000:3.0:1788845027.428563:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 860 00000100:00100000:3.0:1788845027.428573:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641500672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845027.428615:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845027.428637:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac909 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81f7 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.428678:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845027.428722:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660229376:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845027.428821:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845027.432729:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845027 00000004:00010000:2.0:1788845027.432739:0:11036:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 648 -> 776 00000100:00100000:2.0:1788845027.432766:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660229376:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845027.432793:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845027.432816:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac909 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.432840:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.432843:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845027.432848:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960600/0xb2187301287ac909 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.432863:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960600, rc 0) 00000100:00100000:2.0:1788845027.432984:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641500672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4412us (4852us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.432993:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 860 00000100:00100000:2.0:1788845027.437028:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.437226:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641500800 00000100:00100000:1.0:1788845027.437237:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641500800 00000100:00100000:1.0:1788845027.437301:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 861 00000100:00100000:1.0:1788845027.437312:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641500800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.437348:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.437370:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3200/0xb2187301287ac910 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a81fe expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.437410:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.437448:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660229504:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845027.437539:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.443572:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660229504:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845027.443604:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845027.443624:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3200/0xb2187301287ac910 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81fe expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.443668:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.443671:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.443676:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3200/0xb2187301287ac910 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81fe expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.443691:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3200, rc 0) 00000100:00100000:1.0:1788845027.443766:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641500800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6456us (6748us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845027.443775:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 861 00000100:00100000:3.0:1788845027.449929:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.450029:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641500928 00000100:00100000:0.0:1788845027.450040:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641500928 00000100:00100000:0.0:1788845027.450100:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 568 00000100:00100000:0.0:1788845027.450111:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641500928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845027.450130:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845027.450148:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8205 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.450193:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.450204:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.450212:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:0.0:1788845027.450217:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:0.0:1788845027.450222:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1273856->1318911] (req 1273856->1318911) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.450257:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.450278:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 9 pid: 6682 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.450293:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660229632:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845027.450420:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.450430:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 11 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.450440:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660229760:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845027.450463:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.459931:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.459975:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.460060:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641501056 00000100:00100000:1.0:1788845027.460069:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641501056 00010000:00010000:1.0:1788845027.460108:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 12 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.460147:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641501184 00000100:00100000:1.0:1788845027.460150:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641501184 00010000:00010000:1.0:1788845027.460165:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 10 pid: 6682 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.460194:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 316 00000100:00100000:1.0:1788845027.460204:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641501056:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845027.460220:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845027.460242:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 12 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.460276:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 12 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.460304:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845027.460371:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641501056:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 168us (494us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845027.460379:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 316 00000100:00100000:1.0:1788845027.460398:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 317 00000100:00100000:1.0:1788845027.460405:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641501184:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845027.460412:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845027.460421:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 10 pid: 6682 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.460439:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 10 pid: 6682 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.460452:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:1.0:1788845027.460457:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 5) 00010000:00010000:1.0:1788845027.460465:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.460476:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.460524:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.460539:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660229888:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845027.460588:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00010000:00010000:0.0:1788845027.461082:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac8f4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81e9 expref: 10 pid: 6682 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.461098:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660229632:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845027.461111:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a23fd2a/0xb2187301287ac8ed lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1081343) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81db expref: 11 pid: 6685 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.461120:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660229760:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845027.461144:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.461156:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3c00, rc 0) 00000100:00100000:0.0:1788845027.461209:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641500928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11100us (11327us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845027.461216:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 568 00000100:00100000:1.0:1788845027.467151:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660229888:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845027.467184:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845027.467241:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641501184:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6836us (7265us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845027.467253:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 317 00000100:00100000:3.0:1788845027.468441:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.468600:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641501312 00000100:00100000:2.0:1788845027.468610:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641501312 00002000:00100000:2.0:1788845027.468659:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a0360b6a00 x1875739641501312/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.468671:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b6a00 with x1875739641501312 ext(1273856->1277951) 00002000:00010000:2.0:1788845027.468677:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b6a00 00000100:00100000:2.0:1788845027.468707:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 332 00000100:00100000:2.0:1788845027.468717:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641501312:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.469017:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641501312:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 300us (580us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.469025:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 332 00002000:00100000:2.0:1788845027.469032:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a0360b6a00 x1875739641501312/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.469041:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b6a00 with x1875739641501312 ext(1273856->1277951) 00002000:00010000:2.0:1788845027.469045:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b6a00 00000100:00100000:3.0:1788845027.478509:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.478665:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641501440 00000100:00100000:1.0:1788845027.478676:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641501440 00002000:00100000:1.0:1788845027.478730:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1314816->1318911) req@ffff94a0360b5f80 x1875739641501440/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.478741:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5f80 with x1875739641501440 ext(1314816->1318911) 00002000:00010000:1.0:1788845027.478747:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5f80 00000100:00100000:1.0:1788845027.478787:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 333 00000100:00100000:1.0:1788845027.478797:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641501440:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.479098:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641501440:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 301us (599us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.479107:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 333 00002000:00100000:1.0:1788845027.479114:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1314816->1318911) req@ffff94a0360b5f80 x1875739641501440/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.479142:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5f80 with x1875739641501440 ext(1314816->1318911) 00002000:00010000:1.0:1788845027.479146:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5f80 00000100:00100000:1.0:1788845027.495365:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.495428:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.495537:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641501568 00000100:00100000:0.0:1788845027.495561:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641501568 00000100:00100000:0.0:1788845027.495634:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 862 00000100:00100000:0.0:1788845027.495645:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641501568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845027.495668:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845027.495690:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3600/0xb2187301287ac91e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a820c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.495730:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.495772:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660230016:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845027.495858:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.496337:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641501696 00000100:00100000:2.0:1788845027.496343:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641501696 00000100:00100000:2.0:1788845027.496384:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 569 00000100:00100000:2.0:1788845027.496393:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641501696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.496404:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.496417:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac925 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8213 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.496468:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660230144:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.496503:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845027.499527:0:7983:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845026 -> 1788845027 00002000:00010000:2.0:1788845027.499539:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845026 -> 1788845027 00002000:00010000:2.0:1788845027.499541:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845026 -> 1788845027 00000100:00100000:2.0:1788845027.499566:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660230144:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845027.499603:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac925 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8213 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.499628:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffa10918/0xb2187301287ac925 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8213 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.499641:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0a00, rc 0) 00000100:00100000:2.0:1788845027.499702:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641501696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3310us (4273us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.499710:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 569 00000100:00100000:0.0:1788845027.500043:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660230016:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845027.500059:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:0.0:1788845027.500071:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3600/0xb2187301287ac91e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a820c expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.500085:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845027.500088:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845027.500092:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c3600/0xb2187301287ac91e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a820c expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.500104:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3600, rc 0) 00000100:00100000:0.0:1788845027.500142:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641501568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4498us (4786us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845027.500149:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 862 00000100:00100000:3.0:1788845027.507503:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.507605:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641501824 00000100:00100000:2.0:1788845027.507616:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641501824 00000100:00100000:2.0:1788845027.507676:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 863 00000100:00100000:2.0:1788845027.507685:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641501824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.507704:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.507722:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac92c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a821a expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.507771:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.507810:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660230272:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.507931:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.517540:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.517759:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660230272:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845027.517801:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:0.0:1788845027.517822:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac92c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a821a expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.517843:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845027.517845:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845027.517850:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac92c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a821a expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.517867:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0a00, rc 0) 00000100:00100000:0.0:1788845027.517975:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641501824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10290us (10479us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845027.517984:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 863 00000100:00100000:2.0:1788845027.518074:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641501952 00000100:00100000:2.0:1788845027.518081:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641501952 00000100:00100000:2.0:1788845027.518130:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 570 00000100:00100000:2.0:1788845027.518140:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641501952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.518155:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.518175:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac933 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8221 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.518228:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cedf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660230400:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.518284:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.529386:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cedf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660230400:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845027.529436:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac933 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8221 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.529462:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287ac933 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8221 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.529476:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845027.529543:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641501952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11403us (12012us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.529568:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 570 00000100:00100000:3.0:1788845027.533855:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.533952:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.534004:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641502208 00000100:00100000:0.0:1788845027.534014:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641502208 00000100:00100000:0.0:1788845027.534082:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 571 00000100:00100000:0.0:1788845027.534093:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641502208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845027.534112:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845027.534131:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000087ae7707/0xb2187301287ac93a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a822f expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.534190:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660230528:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845027.534283:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.535032:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641502080 00000100:00100000:2.0:1788845027.535039:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641502080 00000100:00100000:2.0:1788845027.535078:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 864 00000100:00100000:2.0:1788845027.535085:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641502080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.535102:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.535119:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac941 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8228 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.535152:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.535193:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cedf80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660230656:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.535250:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845027.541139:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660230528:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845027.541469:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000087ae7707/0xb2187301287ac93a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a822f expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.541522:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000087ae7707/0xb2187301287ac93a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a822f expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.541537:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c3600, rc 0) 00000100:00100000:0.0:1788845027.541619:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641502208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7527us (7666us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845027.541629:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 571 00000100:00100000:2.0:1788845027.541986:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cedf80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660230656:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845027.542002:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845027.542014:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac941 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8228 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.542027:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.542030:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845027.542035:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287ac941 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8228 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.542045:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0a00, rc 0) 00000100:00100000:2.0:1788845027.542082:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641502080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6997us (8241us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.542096:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 864 00000100:00100000:3.0:1788845027.549193:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.550296:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641502336 00000100:00100000:1.0:1788845027.550308:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641502336 00000100:00100000:1.0:1788845027.550393:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 865 00000100:00100000:1.0:1788845027.550404:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641502336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.550426:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.550448:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8236 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.550479:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.550489:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.550524:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.550538:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 10 pid: 5841 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.550549:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660230784:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845027.550651:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845027.554803:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.555066:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641502464 00000100:00100000:2.0:1788845027.555075:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641502464 00010000:00010000:2.0:1788845027.555128:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 11 pid: 5841 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.555186:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 318 00000100:00100000:2.0:1788845027.555197:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641502464:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845027.555212:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845027.555242:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 11 pid: 5841 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.555276:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 11 pid: 5841 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.555518:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660230784:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845027.555533:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2e00/0xb2187301287ac902 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a81f0 expref: 11 pid: 5841 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.555553:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.555554:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.555559:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.555571:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.555665:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.557860:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.557871:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.557876:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3000, rc 0) 00000100:00100000:1.0:1788845027.558008:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641502336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7601us (8819us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845027.558018:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 865 00010000:00010000:3.0:1788845027.567963:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845027.567991:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660230912:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845027.568211:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.571853:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660230912:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845027.571922:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845027.571987:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641502464:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 16790us (17197us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845027.571999:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 318 00000100:00100000:3.0:1788845027.575766:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.575879:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641502592 00000100:00100000:2.0:1788845027.575926:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641502592 00002000:00100000:2.0:1788845027.575994:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532ad80 x1875739641502592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.576006:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532ad80 with x1875739641502592 00000100:00100000:2.0:1788845027.576045:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:2.0:1788845027.576055:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641502592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.576286:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641502592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 232us (529us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.576295:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 539 00002000:00100000:2.0:1788845027.576302:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532ad80 x1875739641502592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.576310:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532ad80 with x1875739641502592 00000100:00100000:0.0:1788845027.584646:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.585007:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641502720 00000100:00100000:3.0:1788845027.585019:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641502720 00002000:00100000:3.0:1788845027.585073:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db8380 x1875739641502720/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845027.585084:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db8380 with x1875739641502720 00000100:00100000:3.0:1788845027.585153:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 540 00000100:00100000:3.0:1788845027.585162:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641502720:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845027.585393:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641502720:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (756us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845027.585400:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 540 00002000:00100000:3.0:1788845027.585407:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db8380 x1875739641502720/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845027.585414:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db8380 with x1875739641502720 00000100:00100000:3.0:1788845027.591012:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.591239:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641502848 00000100:00100000:1.0:1788845027.591252:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641502848 00002000:00100000:1.0:1788845027.591316:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035328e00 x1875739641502848/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.591335:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328e00 with x1875739641502848 00000100:00100000:1.0:1788845027.591376:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 541 00000100:00100000:1.0:1788845027.591388:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641502848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.591662:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641502848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 275us (659us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.591672:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 541 00002000:00100000:1.0:1788845027.591679:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035328e00 x1875739641502848/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.591689:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328e00 with x1875739641502848 00000100:00100000:0.0:1788845027.600804:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.600912:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641502976 00000100:00100000:3.0:1788845027.600922:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641502976 00002000:00100000:3.0:1788845027.600987:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db9c00 x1875739641502976/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 12776/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845027.600999:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db9c00 with x1875739641502976 00000100:00100000:3.0:1788845027.601045:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 542 00000100:00100000:3.0:1788845027.601054:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641502976:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845027.601554:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:3.0:1788845027.601559:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845027 -> 1788845027 00000004:00010000:3.0:1788845027.601562:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845027 -> 1788845026 00000004:00010000:3.0:1788845027.601564:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845027 -> 1788845027 00000004:00010000:3.0:1788845027.601566:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:3.0:1788845027.601650:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641502976:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 596us (857us total) trans 4294967643 rc 0/0 00000100:00100000:3.0:1788845027.601658:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 542 00002000:00100000:3.0:1788845027.601670:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db9c00 x1875739641502976/t4294967643(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 12776/448 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845027.601679:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db9c00 with x1875739641502976 00000100:00100000:0.0:1788845027.612193:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.612399:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641503104 00000100:00100000:1.0:1788845027.612425:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641503104 00000100:00100000:1.0:1788845027.612504:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 866 00000100:00100000:1.0:1788845027.612514:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641503104:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:2.0:1788845027.693280:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967643 is committed 00000100:00100000:3.0:1788845027.693544:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641503104:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 81027us (81355us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845027.693557:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 866 00000100:00100000:0.0:1788845027.697502:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.697755:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641503232 00000100:00100000:1.0:1788845027.697765:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641503232 00000100:00100000:1.0:1788845027.697827:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 572 00000100:00100000:1.0:1788845027.697837:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641503232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.697856:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.697874:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aebd45ad/0xb2187301287ac94f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a823d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.697966:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660231040:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845027.698050:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.702286:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660231040:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845027.702336:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aebd45ad/0xb2187301287ac94f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a823d expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.702360:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aebd45ad/0xb2187301287ac94f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a823d expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.702374:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2c00, rc 0) 00000100:00100000:1.0:1788845027.702439:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641503232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4602us (4945us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845027.702448:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 572 00000100:00100000:0.0:1788845027.711097:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.713959:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641503360 00000100:00100000:3.0:1788845027.713973:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641503360 00000100:00100000:3.0:1788845027.714037:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 573 00000100:00100000:3.0:1788845027.714050:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641503360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845027.714070:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845027.714090:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac956 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8244 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845027.714683:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660231168:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845027.714778:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845027.718710:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660231168:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845027.718749:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac956 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8244 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.718775:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac956 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8244 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.718791:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961c00, rc 0) 00000100:00100000:3.0:1788845027.718865:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641503360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4817us (7775us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845027.720685:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 573 00000100:00100000:2.0:1788845027.725700:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845027.725767:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641503488 00000100:00100000:0.0:1788845027.725775:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641503488 00000100:00100000:0.0:1788845027.725869:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 867 00000100:00100000:0.0:1788845027.725877:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641503488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845027.725922:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845027.725950:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287ac95d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a824b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.725992:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845027.726031:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660231296:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845027.726311:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000004:00010000:2.0:1788845027.734965:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845027 00000100:00100000:2.0:1788845027.734995:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660231296:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845027.735036:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845027.735057:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287ac95d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a824b expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.735077:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.735080:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845027.735085:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287ac95d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a824b expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.735291:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2e00, rc 0) 00000100:00100000:2.0:1788845027.735357:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641503488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9479us (9663us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.735365:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 867 00000100:00100000:0.0:1788845027.739626:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.739728:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641503616 00000100:00100000:2.0:1788845027.739738:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641503616 00000100:00100000:2.0:1788845027.739833:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 868 00000100:00100000:2.0:1788845027.739844:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641503616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.739864:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.740014:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac964 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8252 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.740051:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.740088:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660231424:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.740291:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.745953:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660231424:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845027.746001:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845027.746022:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac964 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8252 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.746041:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.746044:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845027.746048:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287ac964 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8252 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.746062:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845027.746465:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641503616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6610us (6837us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.746474:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 868 00000100:00100000:0.0:1788845027.750588:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.750707:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641503744 00000100:00100000:1.0:1788845027.750729:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641503744 00002000:00100000:1.0:1788845027.750779:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1392640->1396735) req@ffff949f01db9880 x1875739641503744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.750790:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db9880 with x1875739641503744 ext(1392640->1396735) 00002000:00010000:1.0:1788845027.750796:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db9880 00000100:00100000:1.0:1788845027.750996:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 334 00000100:00100000:1.0:1788845027.751007:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641503744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.751514:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641503744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 507us (934us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.751523:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 334 00002000:00100000:1.0:1788845027.751531:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1392640->1396735) req@ffff949f01db9880 x1875739641503744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.751540:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db9880 with x1875739641503744 ext(1392640->1396735) 00002000:00010000:1.0:1788845027.751544:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db9880 00000100:00100000:0.0:1788845027.755439:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.755524:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641503872 00000100:00100000:2.0:1788845027.755534:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641503872 00002000:00100000:2.0:1788845027.755581:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1409024->1413119) req@ffff949f01db8000 x1875739641503872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.755593:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db8000 with x1875739641503872 ext(1409024->1413119) 00002000:00010000:2.0:1788845027.755599:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db8000 00000100:00100000:2.0:1788845027.755633:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 335 00000100:00100000:2.0:1788845027.755643:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641503872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.755946:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641503872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 295us (508us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.755954:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 335 00002000:00100000:2.0:1788845027.755962:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1409024->1413119) req@ffff949f01db8000 x1875739641503872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.755991:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db8000 with x1875739641503872 ext(1409024->1413119) 00002000:00010000:2.0:1788845027.755995:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db8000 00000100:00100000:0.0:1788845027.760931:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.761044:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641504000 00000100:00100000:2.0:1788845027.761055:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641504000 00000100:00100000:2.0:1788845027.761110:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 574 00000100:00100000:2.0:1788845027.761119:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641504000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.761138:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.761158:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac96b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8259 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.761223:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660231552:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.761319:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.764230:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660231552:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845027.764271:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac96b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8259 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.764298:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287ac96b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8259 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.764311:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845027.764373:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641504000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3255us (3502us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.764381:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 574 00000100:00100000:0.0:1788845027.768189:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.768292:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641504128 00000100:00100000:3.0:1788845027.768302:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641504128 00000100:00100000:3.0:1788845027.768364:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 575 00000100:00100000:3.0:1788845027.768375:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641504128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845027.768394:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845027.768413:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac972 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8260 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845027.768485:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660231680:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845027.768569:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845027.775042:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660231680:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845027.775085:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac972 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8260 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.775111:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac972 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8260 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.775126:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961c00, rc 0) 00000100:00100000:3.0:1788845027.775194:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641504128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6820us (7014us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845027.775203:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 575 00000100:00100000:0.0:1788845027.787081:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.787383:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641504256 00000100:00100000:1.0:1788845027.787393:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641504256 00002000:00100000:1.0:1788845027.787444:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1327104->1331199) req@ffff949f01dbb100 x1875739641504256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.787469:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01dbb100 with x1875739641504256 ext(1327104->1331199) 00002000:00010000:1.0:1788845027.787475:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01dbb100 00000100:00100000:1.0:1788845027.787510:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 336 00000100:00100000:1.0:1788845027.787519:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641504256:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.787769:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641504256:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (698us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.787792:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 336 00002000:00100000:1.0:1788845027.787800:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1327104->1331199) req@ffff949f01dbb100 x1875739641504256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.787821:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01dbb100 with x1875739641504256 ext(1327104->1331199) 00002000:00010000:1.0:1788845027.787825:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01dbb100 00000100:00100000:0.0:1788845027.793329:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.794015:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641504384 00000100:00100000:1.0:1788845027.794027:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641504384 00002000:00100000:1.0:1788845027.794078:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1388544->1392639) req@ffff949f01db9f80 x1875739641504384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.794091:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db9f80 with x1875739641504384 ext(1388544->1392639) 00002000:00010000:1.0:1788845027.794096:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db9f80 00000100:00100000:1.0:1788845027.794133:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 337 00000100:00100000:1.0:1788845027.794143:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641504384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.794442:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641504384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 298us (1134us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.794449:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 337 00002000:00100000:1.0:1788845027.794466:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1388544->1392639) req@ffff949f01db9f80 x1875739641504384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.794476:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db9f80 with x1875739641504384 ext(1388544->1392639) 00002000:00010000:1.0:1788845027.794480:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db9f80 00000100:00100000:0.0:1788845027.799004:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.799551:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641504512 00000100:00100000:1.0:1788845027.799562:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641504512 00000100:00100000:1.0:1788845027.799929:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 869 00000100:00100000:1.0:1788845027.799949:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641504512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.799988:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.800010:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac979 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8267 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.800047:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.800089:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660231808:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845027.800162:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.802591:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660231808:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845027.802627:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845027.802646:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac979 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8267 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.802666:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.802669:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.802674:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac979 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8267 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.802691:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2c00, rc 0) 00000100:00100000:1.0:1788845027.802760:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641504512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2813us (3763us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845027.802768:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 869 00000100:00100000:0.0:1788845027.806201:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.806308:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641504640 00000100:00100000:3.0:1788845027.806318:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641504640 00000100:00100000:3.0:1788845027.806395:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 870 00000100:00100000:3.0:1788845027.806407:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641504640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845027.806429:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845027.806453:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac980 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a826e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.806505:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845027.806550:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660231936:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845027.806639:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845027.810154:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660231936:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845027.810181:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845027.810201:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac980 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a826e expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.810225:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845027.810229:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845027.810235:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287ac980 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a826e expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.810251:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961c00, rc 0) 00000100:00100000:3.0:1788845027.810312:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641504640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3907us (4120us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845027.810321:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 870 00000100:00100000:0.0:1788845027.813169:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.813285:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641504768 00000100:00100000:2.0:1788845027.813309:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641504768 00000100:00100000:2.0:1788845027.813419:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 871 00000100:00100000:2.0:1788845027.813429:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641504768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.813451:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.813481:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac987 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8275 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.813517:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.813557:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660232064:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.813637:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.816125:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660232064:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845027.816149:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845027.816167:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac987 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8275 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.816187:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.816191:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845027.816196:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac987 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8275 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.816210:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:2.0:1788845027.816285:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641504768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2858us (3125us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.816293:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 871 00000100:00100000:0.0:1788845027.820020:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.820128:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641504896 00000100:00100000:2.0:1788845027.820151:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641504896 00000100:00100000:2.0:1788845027.820212:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 872 00000100:00100000:2.0:1788845027.820222:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641504896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.820246:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.820269:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac98e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a827c expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.820306:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.820347:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660232192:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.820424:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.823391:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660232192:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845027.823421:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845027.823443:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac98e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a827c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.823476:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.823479:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845027.823484:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac98e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a827c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.823499:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:2.0:1788845027.823559:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641504896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3337us (3548us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.823567:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 872 00000100:00100000:0.0:1788845027.828174:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.828256:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641505024 00000100:00100000:1.0:1788845027.828267:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641505024 00000100:00100000:1.0:1788845027.828332:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 576 00000100:00100000:1.0:1788845027.828342:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641505024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.828374:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.828392:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aebd45ad/0xb2187301287ac995 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8283 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.828450:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660232320:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845027.828544:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.832144:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660232320:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845027.832192:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aebd45ad/0xb2187301287ac995 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8283 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.832220:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aebd45ad/0xb2187301287ac995 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8283 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.832235:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2c00, rc 0) 00000100:00100000:1.0:1788845027.832298:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641505024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3957us (4134us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845027.832308:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 576 00000100:00100000:0.0:1788845027.835654:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.835755:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641505152 00000100:00100000:2.0:1788845027.835780:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641505152 00000100:00100000:2.0:1788845027.835847:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 577 00000100:00100000:2.0:1788845027.835858:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641505152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.835876:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.835942:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ac99c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a828a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.835999:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660232448:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.836077:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.840646:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660232448:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845027.840687:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ac99c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a828a expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.840714:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ac99c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a828a expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.840728:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:2.0:1788845027.840791:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641505152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4935us (5147us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.840800:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 577 00000100:00100000:0.0:1788845027.846155:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.846605:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641505280 00000100:00100000:1.0:1788845027.846617:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641505280 00002000:00100000:1.0:1788845027.846667:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684380 x1875739641505280/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.846677:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684380 with x1875739641505280 00000100:00100000:1.0:1788845027.846766:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 543 00000100:00100000:1.0:1788845027.846778:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641505280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.847147:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641505280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 370us (1000us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.847155:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 543 00002000:00100000:1.0:1788845027.847162:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684380 x1875739641505280/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.847170:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684380 with x1875739641505280 00000100:00100000:0.0:1788845027.850064:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.850777:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641505408 00000100:00100000:2.0:1788845027.850790:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641505408 00002000:00100000:2.0:1788845027.850856:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685c00 x1875739641505408/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.850868:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685c00 with x1875739641505408 00000100:00100000:2.0:1788845027.850950:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 544 00000100:00100000:2.0:1788845027.850961:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641505408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.851186:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641505408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 226us (1133us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.851193:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 544 00002000:00100000:2.0:1788845027.851200:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685c00 x1875739641505408/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.851208:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685c00 with x1875739641505408 00000100:00100000:0.0:1788845027.854878:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.855044:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641505536 00000100:00100000:1.0:1788845027.855054:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641505536 00000100:00100000:1.0:1788845027.855118:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 873 00000100:00100000:1.0:1788845027.855128:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641505536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.855152:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.855172:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac9a3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8291 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.855211:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.855252:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660232576:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845027.855339:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.859656:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660232576:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845027.859688:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845027.859718:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac9a3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8291 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.859740:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.859757:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.859764:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac9a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8291 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.859780:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2c00, rc 0) 00000100:00100000:1.0:1788845027.859849:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641505536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4722us (4979us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845027.859857:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 873 00000100:00100000:3.0:1788845027.864406:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.864494:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641505664 00000100:00100000:1.0:1788845027.864504:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641505664 00000100:00100000:1.0:1788845027.864562:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 874 00000100:00100000:1.0:1788845027.864571:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641505664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.864591:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.864612:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3400/0xb2187301287ac9aa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8298 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.864647:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.864683:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660232704:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845027.864761:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.868264:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660232704:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845027.868370:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845027.868391:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3400/0xb2187301287ac9aa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8298 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.868413:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.868427:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.868433:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3400/0xb2187301287ac9aa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8298 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.868449:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3400, rc 0) 00000100:00100000:1.0:1788845027.868578:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641505664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4007us (4183us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845027.868587:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 874 00000100:00100000:0.0:1788845027.874707:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.875016:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641505792 00000100:00100000:2.0:1788845027.875027:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641505792 00000100:00100000:2.0:1788845027.875258:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 875 00000100:00100000:2.0:1788845027.875271:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641505792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.875292:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.875314:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac9b1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a829f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.875430:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.875493:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660232832:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.875584:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.879659:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660232832:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845027.879690:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845027.879711:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac9b1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a829f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.879755:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.879759:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845027.879764:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287ac9b1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a829f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.879779:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:2.0:1788845027.879917:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641505792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4648us (5218us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.879926:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 875 00000100:00100000:0.0:1788845027.885062:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.885309:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641505920 00000100:00100000:2.0:1788845027.885327:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641505920 00002000:00100000:2.0:1788845027.885380:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1318912->1323007) req@ffff94a035684e00 x1875739641505920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.885392:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035684e00 with x1875739641505920 ext(1318912->1323007) 00002000:00010000:2.0:1788845027.885397:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035684e00 00000100:00100000:2.0:1788845027.885665:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 338 00000100:00100000:2.0:1788845027.885676:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641505920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.885977:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641505920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 301us (924us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.885985:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 338 00002000:00100000:2.0:1788845027.885993:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1318912->1323007) req@ffff94a035684e00 x1875739641505920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.886002:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035684e00 with x1875739641505920 ext(1318912->1323007) 00002000:00010000:2.0:1788845027.886021:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035684e00 00000100:00100000:0.0:1788845027.890980:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.891071:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641506048 00000100:00100000:1.0:1788845027.891082:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641506048 00002000:00100000:1.0:1788845027.891134:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1323008->1327103) req@ffff94a035686d80 x1875739641506048/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.891145:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035686d80 with x1875739641506048 ext(1323008->1327103) 00002000:00010000:1.0:1788845027.891286:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035686d80 00000100:00100000:1.0:1788845027.891330:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 339 00000100:00100000:1.0:1788845027.891342:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641506048:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845027.891715:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641506048:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 375us (743us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845027.891723:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 339 00002000:00100000:1.0:1788845027.891732:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1323008->1327103) req@ffff94a035686d80 x1875739641506048/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845027.891740:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035686d80 with x1875739641506048 ext(1323008->1327103) 00002000:00010000:1.0:1788845027.891744:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035686d80 00000100:00100000:3.0:1788845027.896001:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.896384:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641506176 00000100:00100000:2.0:1788845027.896395:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641506176 00000100:00100000:2.0:1788845027.896566:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 578 00000100:00100000:2.0:1788845027.896579:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641506176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.896598:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.896618:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ac9b8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82a6 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.896678:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660232960:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.896770:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845027.899972:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660232960:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845027.900014:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ac9b8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82a6 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.900040:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d0cf6440/0xb2187301287ac9b8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82a6 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.900054:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:2.0:1788845027.900115:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641506176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3537us (4121us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845027.900124:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 578 00000100:00100000:0.0:1788845027.903561:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845027.903615:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641506304 00000100:00100000:3.0:1788845027.903624:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641506304 00000100:00100000:3.0:1788845027.903676:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 579 00000100:00100000:3.0:1788845027.903686:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641506304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845027.903706:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845027.903722:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac9bf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82ad expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845027.903771:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660233088:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845027.903850:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845027.907543:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660233088:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845027.907584:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac9bf lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ad expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.907611:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287ac9bf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ad expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.907646:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961c00, rc 0) 00000100:00100000:0.0:1788845027.908027:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641506304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4340us (4476us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845027.908036:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 579 00000100:00100000:0.0:1788845027.916167:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.916465:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641506432 00000100:00100000:2.0:1788845027.916475:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641506432 00000100:00100000:2.0:1788845027.916535:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 876 00000100:00100000:2.0:1788845027.916546:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641506432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.916581:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.916602:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac9c6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82b4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.916640:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.916679:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660233216:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845027.916766:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845027.920822:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660233216:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845027.920851:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845027.920870:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac9c6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82b4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.920923:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845027.920927:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845027.920941:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287ac9c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82b4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845027.920957:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:3.0:1788845027.921007:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641506432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4463us (4860us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845027.921016:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 876 00010000:00010000:3.0:1788845027.922433:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845027.925035:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.925454:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641506560 00000100:00100000:1.0:1788845027.925531:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641506560 00000100:00100000:1.0:1788845027.925670:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 877 00000100:00100000:1.0:1788845027.925681:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641506560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.925702:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.925723:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3400/0xb2187301287ac9cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82bb expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.925759:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.925795:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660233344:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845027.925910:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.928274:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660233344:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845027.928304:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845027.928323:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3400/0xb2187301287ac9cd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82bb expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.928345:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.928348:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.928353:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3400/0xb2187301287ac9cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82bb expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.928368:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3400, rc 0) 00000100:00100000:1.0:1788845027.928469:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641506560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2789us (3443us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845027.928478:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 877 00000100:00100000:0.0:1788845027.932867:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845027.933064:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641506688 00000100:00100000:1.0:1788845027.933074:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641506688 00000100:00100000:1.0:1788845027.933148:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 878 00000100:00100000:1.0:1788845027.933169:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641506688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845027.933192:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845027.933215:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac9d4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82c2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.933272:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845027.933313:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660233472:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845027.933393:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845027.937014:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660233472:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845027.937044:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845027.937060:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac9d4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c2 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.937080:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.937083:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.937088:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2c00/0xb2187301287ac9d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c2 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.937102:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f2c00, rc 0) 00000100:00100000:1.0:1788845027.937252:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641506688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4084us (4395us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845027.937270:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 878 00000100:00100000:0.0:1788845027.940116:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.940411:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641506816 00000100:00100000:2.0:1788845027.940422:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641506816 00000100:00100000:2.0:1788845027.940563:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 879 00000100:00100000:2.0:1788845027.940574:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641506816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845027.940595:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845027.940614:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82c9 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.940646:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.940656:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.940686:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.940702:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 14 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.940727:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cedc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660233600:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845027.940841:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845027.947464:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.947605:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641506944 00000100:00100000:2.0:1788845027.947625:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641506944 00002000:00100000:2.0:1788845027.947678:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358e680 x1875739641506944/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.947690:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358e680 with x1875739641506944 00010000:00010000:2.0:1788845027.947703:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 15 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.947723:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 15 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845027.947731:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0358e680 00000100:00100000:2.0:1788845027.947766:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 20 00000100:00100000:2.0:1788845027.947776:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641506944:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845027.948913:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cedc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660233600:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845027.948961:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.948994:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845027.948997:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845027.949003:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845027.949013:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0e00, rc 0) 00000100:00100000:1.0:1788845027.949114:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641506816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8541us (9005us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845027.949123:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 879 00000004:00010000:0.0:1788845027.953521:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:0.0:1788845027.953531:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845027 -> 1788845027 00000004:00010000:0.0:1788845027.953534:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845027 -> 1788845026 00000004:00010000:0.0:1788845027.953536:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845027 -> 1788845027 00000004:00010000:0.0:1788845027.953538:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:0.0:1788845027.953676:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641506944:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5896us (6227us total) trans 4294967644 rc 0/0 00000100:00100000:0.0:1788845027.953687:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 20 00002000:00100000:0.0:1788845027.953699:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358e680 x1875739641506944/t4294967644(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:133/0 lens 488/448 e 0 to 0 dl 1788845038 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845027.953710:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358e680 with x1875739641506944 00010000:00010000:0.0:1788845027.953720:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 17 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845027.953740:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 17 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845027.953762:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0358e680 00000100:00100000:0.0:1788845027.957347:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.957473:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641507072 00000100:00100000:2.0:1788845027.957484:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641507072 00010000:00010000:2.0:1788845027.957527:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 17 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.957570:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 319 00000100:00100000:2.0:1788845027.957581:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641507072:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845027.957597:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845027.957643:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 17 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.957659:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 17 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.957680:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f3000/0xb2187301287ac948 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8236 expref: 17 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.957696:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845027.957702:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.957711:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.957719:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845027.957753:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845027.957765:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cef480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660233728:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845027.957857:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845027.961227:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.961476:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cef480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660233728:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845027.961506:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845027.961565:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641507072:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3985us (4229us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845027.961584:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 319 00000100:00100000:2.0:1788845027.962853:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641507200 00000100:00100000:2.0:1788845027.962866:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641507200 00002000:00100000:2.0:1788845027.962945:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532b800 x1875739641507200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.962957:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532b800 with x1875739641507200 00000100:00100000:2.0:1788845027.962988:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 545 00000100:00100000:2.0:1788845027.962998:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641507200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.963526:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641507200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 527us (2308us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.963535:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 545 00002000:00100000:2.0:1788845027.963543:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532b800 x1875739641507200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.963552:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532b800 with x1875739641507200 00000100:00100000:3.0:1788845027.966527:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.967179:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641507328 00000100:00100000:2.0:1788845027.967191:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641507328 00002000:00100000:2.0:1788845027.967246:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03455f80 x1875739641507328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.967258:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455f80 with x1875739641507328 00000100:00100000:2.0:1788845027.967285:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 546 00000100:00100000:2.0:1788845027.967295:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641507328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.967737:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641507328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 440us (1217us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.967746:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 546 00002000:00100000:2.0:1788845027.967767:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03455f80 x1875739641507328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.967776:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455f80 with x1875739641507328 00000100:00100000:3.0:1788845027.974319:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.974401:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641507456 00000100:00100000:2.0:1788845027.974412:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641507456 00002000:00100000:2.0:1788845027.974474:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03455500 x1875739641507456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.974485:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455500 with x1875739641507456 00000100:00100000:2.0:1788845027.974530:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 547 00000100:00100000:2.0:1788845027.974539:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641507456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.974805:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641507456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (494us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.974818:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 547 00002000:00100000:2.0:1788845027.974825:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03455500 x1875739641507456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.974833:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455500 with x1875739641507456 00000100:00100000:1.0:1788845027.978001:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.978098:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641507584 00000100:00100000:2.0:1788845027.978111:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641507584 00002000:00100000:2.0:1788845027.978177:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035689c00 x1875739641507584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.978188:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035689c00 with x1875739641507584 00000100:00100000:2.0:1788845027.978217:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 548 00000100:00100000:2.0:1788845027.978227:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641507584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.978627:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641507584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 399us (633us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.978650:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 548 00002000:00100000:2.0:1788845027.978659:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035689c00 x1875739641507584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.978667:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035689c00 with x1875739641507584 00000100:00100000:3.0:1788845027.982959:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.983598:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641507712 00000100:00100000:2.0:1788845027.983609:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641507712 00002000:00100000:2.0:1788845027.983659:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454e00 x1875739641507712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.983671:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454e00 with x1875739641507712 00000100:00100000:2.0:1788845027.983711:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 549 00000100:00100000:2.0:1788845027.983722:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641507712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845027.984033:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641507712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 312us (1085us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845027.984041:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 549 00002000:00100000:2.0:1788845027.984047:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454e00 x1875739641507712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/4536 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.984054:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454e00 with x1875739641507712 00000100:00100000:3.0:1788845027.986988:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.987136:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641507840 00000100:00100000:2.0:1788845027.987145:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641507840 00002000:00100000:2.0:1788845027.987191:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03457100 x1875739641507840/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/0 e 0 to 0 dl 1788845038 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.987202:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03457100 with x1875739641507840 00000100:00100000:2.0:1788845027.987276:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 550 00000100:00100000:2.0:1788845027.987287:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641507840:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845027.990777:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:2.0:1788845027.990785:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845027 -> 1788845028 00000004:00010000:2.0:1788845027.990786:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845027.990788:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845027 -> 1788845028 00000004:00010000:2.0:1788845027.990790:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:2.0:1788845027.990920:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641507840:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3631us (3937us total) trans 4294967645 rc 0/0 00000100:00100000:2.0:1788845027.990929:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 550 00002000:00100000:2.0:1788845027.990952:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03457100 x1875739641507840/t4294967645(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:133/0 lens 488/448 e 0 to 0 dl 1788845038 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845027.990965:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03457100 with x1875739641507840 00000100:00100000:3.0:1788845027.993217:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845027.993379:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641507968 00000100:00100000:2.0:1788845027.993390:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641507968 00000100:00100000:2.0:1788845027.993510:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 880 00000100:00100000:2.0:1788845027.993521:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641507968:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845028.023021:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967644 is committed 00000001:00080000:1.0:1788845028.023047:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967645 is committed 00000100:00100000:3.0:1788845028.023290:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641507968:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 29765us (30078us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.023304:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 880 00000100:00100000:1.0:1788845028.026903:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.026994:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.027078:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641508096 00000100:00100000:2.0:1788845028.027088:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641508096 00000100:00100000:2.0:1788845028.027149:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 881 00000100:00100000:2.0:1788845028.027160:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641508096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.027188:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.027209:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287ac9e2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82d0 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.027250:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.027320:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cef480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660233856:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.027406:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845028.029524:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845027 00000004:00010000:2.0:1788845028.029534:0:5840:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 776 -> 968 00000100:00100000:2.0:1788845028.029560:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cef480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660233856:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845028.029585:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845028.029607:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287ac9e2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82d0 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.029633:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.029637:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845028.029642:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287ac9e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82d0 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.029659:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:2.0:1788845028.029735:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641508096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2576us (2740us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.029743:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 881 00000100:00100000:0.0:1788845028.031961:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641508224 00000100:00100000:0.0:1788845028.031980:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641508224 00000100:00100000:1.0:1788845028.032177:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 580 00000100:00100000:1.0:1788845028.032191:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641508224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.032214:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.032234:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006ed90043/0xb2187301287ac9e9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82d7 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.032313:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660233984:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.032405:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00002000:00010000:1.0:1788845028.035020:0:6684:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845027 -> 1788845028 00002000:00010000:1.0:1788845028.035032:0:6684:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845027 -> 1788845028 00000100:00100000:1.0:1788845028.035059:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660233984:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.035125:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006ed90043/0xb2187301287ac9e9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82d7 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.035152:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006ed90043/0xb2187301287ac9e9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82d7 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.035168:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f024f3800, rc 0) 00000100:00100000:1.0:1788845028.035248:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641508224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3059us (8369us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.035267:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 580 00000100:00100000:3.0:1788845028.038698:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.038746:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.038856:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641508352 00000100:00100000:0.0:1788845028.038866:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641508352 00000100:00100000:0.0:1788845028.038964:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 882 00000100:00100000:0.0:1788845028.038976:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641508352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845028.039003:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845028.039026:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac9f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82de expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.039068:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.039110:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358f480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660234112:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845028.039189:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.039330:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641508480 00000100:00100000:2.0:1788845028.039335:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641508480 00000100:00100000:2.0:1788845028.039362:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 581 00000100:00100000:2.0:1788845028.039369:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641508480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.039378:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.039389:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac9f7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82e5 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.039444:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660234240:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.039477:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d1e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.042617:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358f480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660234112:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.042652:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845028.042674:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac9f0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82de expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.042695:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.042698:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.042703:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2400/0xb2187301287ac9f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82de expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.042718:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2400, rc 0) 00000100:00100000:1.0:1788845028.042778:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641508352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3804us (4030us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.042786:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 882 00000100:00100000:2.0:1788845028.044321:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660234240:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.044356:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac9f7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82e5 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.044381:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287ac9f7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82e5 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.044397:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1000, rc 0) 00000100:00100000:2.0:1788845028.044464:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641508480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5094us (5777us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.044472:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 581 00000100:00100000:3.0:1788845028.053790:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.053911:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641508608 00000100:00100000:2.0:1788845028.053920:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641508608 00002000:00100000:2.0:1788845028.053998:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03457480 x1875739641508608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.054009:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03457480 with x1875739641508608 00000100:00100000:2.0:1788845028.054039:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 551 00000100:00100000:2.0:1788845028.054048:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641508608:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.054296:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641508608:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 249us (516us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.054306:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 551 00002000:00100000:2.0:1788845028.054328:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03457480 x1875739641508608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.054336:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03457480 with x1875739641508608 00000100:00100000:0.0:1788845028.058413:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.058520:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641508736 00000100:00100000:2.0:1788845028.058530:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641508736 00002000:00100000:2.0:1788845028.058583:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358f100 x1875739641508736/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.058596:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358f100 with x1875739641508736 00000100:00100000:2.0:1788845028.058631:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:2.0:1788845028.058640:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641508736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.058861:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641508736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 221us (460us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.058868:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 552 00002000:00100000:2.0:1788845028.058875:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358f100 x1875739641508736/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.058917:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358f100 with x1875739641508736 00000100:00100000:3.0:1788845028.063080:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.063296:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641508864 00000100:00100000:2.0:1788845028.063307:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641508864 00002000:00100000:2.0:1788845028.063359:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03456680 x1875739641508864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.063371:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03456680 with x1875739641508864 00000100:00100000:2.0:1788845028.063414:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 553 00000100:00100000:2.0:1788845028.063427:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641508864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.063682:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641508864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 258us (612us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.063708:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 553 00002000:00100000:2.0:1788845028.063716:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03456680 x1875739641508864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.063724:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03456680 with x1875739641508864 00000100:00100000:3.0:1788845028.067228:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.067554:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641508992 00000100:00100000:2.0:1788845028.067564:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641508992 00002000:00100000:2.0:1788845028.067616:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454700 x1875739641508992/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.067628:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454700 with x1875739641508992 00000100:00100000:2.0:1788845028.067669:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 554 00000100:00100000:2.0:1788845028.067679:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641508992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.068005:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641508992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 326us (785us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.068014:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 554 00002000:00100000:2.0:1788845028.068021:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454700 x1875739641508992/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.068028:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454700 with x1875739641508992 00000100:00100000:1.0:1788845028.072416:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.072587:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641509120 00000100:00100000:2.0:1788845028.072597:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641509120 00002000:00100000:2.0:1788845028.072652:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035689180 x1875739641509120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.072663:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035689180 with x1875739641509120 00000100:00100000:2.0:1788845028.072704:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 555 00000100:00100000:2.0:1788845028.072716:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641509120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.076405:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641509120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 3678us (3987us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.076421:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 555 00002000:00100000:2.0:1788845028.076433:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035689180 x1875739641509120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.076441:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035689180 with x1875739641509120 00000100:00100000:2.0:1788845028.082740:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.083004:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641509248 00000100:00100000:0.0:1788845028.083014:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641509248 00002000:00100000:0.0:1788845028.083064:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01cef480 x1875739641509248/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.083075:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01cef480 with x1875739641509248 00000100:00100000:0.0:1788845028.083111:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 556 00000100:00100000:0.0:1788845028.083120:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641509248:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845028.083357:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641509248:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 237us (625us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845028.083365:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 556 00002000:00100000:0.0:1788845028.083372:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01cef480 x1875739641509248/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.083381:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01cef480 with x1875739641509248 00000100:00100000:0.0:1788845028.088523:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.088794:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641509376 00000100:00100000:2.0:1788845028.088805:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641509376 00002000:00100000:2.0:1788845028.088872:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d880 x1875739641509376/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.088923:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d880 with x1875739641509376 00000100:00100000:2.0:1788845028.088976:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 557 00000100:00100000:2.0:1788845028.088987:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641509376:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845028.094254:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:2.0:1788845028.094264:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845028 -> 1788845028 00000004:00010000:2.0:1788845028.094267:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845027 -> 1788845026 00000004:00010000:2.0:1788845028.094270:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845028 -> 1788845028 00000004:00010000:2.0:1788845028.094272:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:2.0:1788845028.094357:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641509376:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5367us (5841us total) trans 4294967646 rc 0/0 00000100:00100000:2.0:1788845028.094369:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 557 00002000:00100000:2.0:1788845028.094381:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d880 x1875739641509376/t4294967646(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/448 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.094405:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d880 with x1875739641509376 00000100:00100000:0.0:1788845028.098527:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.098748:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641509504 00000100:00100000:1.0:1788845028.098772:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641509504 00000100:00100000:1.0:1788845028.098850:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 883 00000100:00100000:1.0:1788845028.098861:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641509504:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845028.136192:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967646 is committed 00000100:00100000:2.0:1788845028.136859:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641509504:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 37992us (38335us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.136876:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 883 00000100:00100000:1.0:1788845028.143993:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.144089:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641509632 00000100:00100000:3.0:1788845028.144100:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641509632 00000100:00100000:3.0:1788845028.144164:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 884 00000100:00100000:3.0:1788845028.144176:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641509632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.144198:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.144220:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82ec expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.144251:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.144274:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ec expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.144312:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.144346:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 10 pid: 5839 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.144361:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660234368:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845028.144464:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.148168:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660234368:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845028.148203:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ec expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.148222:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845028.148225:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845028.148230:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ec expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.148240:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961600, rc 0) 00000100:00100000:3.0:1788845028.148307:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641509632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4133us (4323us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.148317:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 884 00000100:00100000:0.0:1788845028.151099:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.151206:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641509760 00000100:00100000:2.0:1788845028.151217:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641509760 00010000:00010000:2.0:1788845028.151261:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 11 pid: 5839 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.151310:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 320 00000100:00100000:2.0:1788845028.151321:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008915450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641509760:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845028.151336:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845028.151364:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 11 pid: 5839 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.151385:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 11 pid: 5839 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.151410:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287ac9db lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a82c9 expref: 11 pid: 5839 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.151428:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.151435:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ec expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.151572:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ec expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.151603:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ec expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.151720:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ec expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.151736:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cefb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660234496:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845028.151836:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.159602:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cefb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660234496:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845028.159755:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845028.159921:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008915450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641509760:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8600us (8831us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.159941:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 320 00000100:00100000:0.0:1788845028.164732:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.165159:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641509888 00000100:00100000:2.0:1788845028.165170:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641509888 00002000:00100000:2.0:1788845028.165227:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ca80 x1875739641509888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.165240:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ca80 with x1875739641509888 00000100:00100000:2.0:1788845028.165285:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 558 00000100:00100000:2.0:1788845028.165296:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641509888:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.180734:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641509888:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 15428us (16003us total) trans 0 rc 1007616/1007616 00000100:00100000:2.0:1788845028.180749:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 558 00002000:00100000:2.0:1788845028.180762:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ca80 x1875739641509888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/440 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 1007616/1007616 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.180785:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ca80 with x1875739641509888 00000100:00100000:0.0:1788845028.185993:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.186241:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641510016 00000100:00100000:2.0:1788845028.186251:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641510016 00000100:00100000:2.0:1788845028.186310:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 582 00000100:00100000:2.0:1788845028.186320:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641510016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.186339:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.186357:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287aca05 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82f3 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.186418:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660234624:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.186672:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00002000:00010000:2.0:1788845028.189413:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845027 -> 1788845028 00000100:00100000:2.0:1788845028.189443:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660234624:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.189494:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287aca05 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82f3 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.189518:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287aca05 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82f3 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.189533:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0e00, rc 0) 00000100:00100000:2.0:1788845028.189600:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641510016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3281us (3619us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.189617:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 582 00000100:00100000:0.0:1788845028.193794:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.193944:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641510144 00000100:00100000:1.0:1788845028.193954:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641510144 00000100:00100000:1.0:1788845028.194009:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 583 00000100:00100000:1.0:1788845028.194019:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641510144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.194042:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.194061:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000b0717d6/0xb2187301287aca0c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a82fa expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.194121:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660234752:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.194197:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.197545:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660234752:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.197586:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000b0717d6/0xb2187301287aca0c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82fa expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.197622:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000b0717d6/0xb2187301287aca0c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82fa expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.197637:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757800, rc 0) 00000100:00100000:1.0:1788845028.197696:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641510144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3677us (3911us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.197721:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 583 00000100:00100000:0.0:1788845028.202118:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.202244:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641510272 00000100:00100000:3.0:1788845028.202262:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641510272 00000100:00100000:3.0:1788845028.202318:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 584 00000100:00100000:3.0:1788845028.202328:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641510272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.202348:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.202366:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8301 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.202410:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.202420:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(1 5) 00010000:00010000:3.0:1788845028.202424:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 5) 00010000:00010000:3.0:1788845028.202429:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1355776->1359871] (req 1355776->1359871) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.202730:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.202749:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 9 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.202764:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660234880:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845028.202940:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.206409:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.206994:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641510400 00000100:00100000:2.0:1788845028.207021:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641510400 00002000:00100000:2.0:1788845028.207073:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1413119) req@ffff94a0359ace00 x1875739641510400/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 504/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.207086:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359ace00 with x1875739641510400 ext(1273856->1413119) 00010000:00010000:2.0:1788845028.207097:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 10 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.207118:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 10 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845028.207126:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a0359ace00 00000100:00100000:2.0:1788845028.207166:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 14 00000100:00100000:2.0:1788845028.207177:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641510400:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845028.207637:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660234880:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845028.207668:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1355776->1359871] (req 1355776->1359871) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.207690:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960a00, rc 0) 00000100:00100000:3.0:1788845028.207724:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641510272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5396us (5615us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.207732:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 584 00000100:00100000:3.0:1788845028.213566:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641510400:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 6378us (7157us total) trans 4294967406 rc 0/0 00000100:00100000:3.0:1788845028.213584:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 14 00002000:00100000:3.0:1788845028.213600:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1413119) req@ffff94a0359ace00 x1875739641510400/t4294967406(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 504/448 e 0 to 0 dl 1788845039 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.213745:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359ace00 with x1875739641510400 ext(1273856->1413119) 00010000:00010000:3.0:1788845028.213758:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.213790:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845028.213799:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a0359ace00 00000100:00100000:0.0:1788845028.218339:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.218461:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641510528 00000100:00100000:2.0:1788845028.218472:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641510528 00010000:00010000:2.0:1788845028.218515:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.218566:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 321 00000100:00100000:2.0:1788845028.218575:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359aea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641510528:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845028.218592:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845028.218879:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.218950:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.218988:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a0f19d6e/0xb2187301287ac917 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1318911) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8205 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.219005:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 5) 00010000:00010000:2.0:1788845028.219009:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 5) 00010000:00010000:2.0:1788845028.219018:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.219028:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.219066:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.219081:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660235008:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845028.219175:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.222860:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660235008:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845028.222917:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845028.222983:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359aea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641510528:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4408us (4655us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845028.222993:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 321 00000100:00100000:0.0:1788845028.225967:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.226182:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641510656 00000100:00100000:1.0:1788845028.226195:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641510656 00000100:00100000:1.0:1788845028.226291:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 340 00000100:00100000:1.0:1788845028.226302:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641510656:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:1.0:1788845028.226763:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1894431 -> 1894431 00002000:00010000:1.0:1788845028.226768:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845028 -> 1788845028 00002000:00010000:1.0:1788845028.226770:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845028 -> 1788845026 00002000:00010000:1.0:1788845028.226773:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845028 -> 1788845027 00002000:00010000:1.0:1788845028.226775:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:1.0:1788845028.226858:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641510656:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 557us (901us total) trans 4294967407 rc 0/0 00000100:00100000:1.0:1788845028.226866:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 340 00000100:00100000:2.0:1788845028.232122:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.232173:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.232259:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641510912 00000100:00100000:3.0:1788845028.232271:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641510912 00000100:00100000:3.0:1788845028.232342:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 585 00000100:00100000:3.0:1788845028.232352:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641510912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.232374:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.232394:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca1a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a830f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.232476:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660235136:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845028.232563:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.233155:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641510784 00000100:00100000:1.0:1788845028.233160:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641510784 00000100:00100000:1.0:1788845028.233203:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 885 00000100:00100000:1.0:1788845028.233212:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641510784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.233224:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.233239:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8308 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.233267:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845028.233279:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845028.233286:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.233313:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.233317:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.233320:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757800, rc 0) 00000100:00100000:1.0:1788845028.233365:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641510784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 153us (1253us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.233372:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 885 00002000:00010000:3.0:1788845028.237044:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845026 -> 1788845028 00002000:00010000:3.0:1788845028.237053:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845027 -> 1788845028 00000100:00100000:3.0:1788845028.237081:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660235136:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845028.237132:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca1a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a830f expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.237158:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca1a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a830f expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.237171:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961800, rc 0) 00000100:00100000:3.0:1788845028.237240:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641510912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4889us (5065us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845028.237248:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 585 00000100:00100000:0.0:1788845028.240936:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.241038:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641511040 00000100:00100000:2.0:1788845028.241048:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641511040 00000100:00100000:2.0:1788845028.241104:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 586 00000100:00100000:2.0:1788845028.241114:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641511040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.241133:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.241152:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287aca28 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8316 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.241231:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ced500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660235264:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.241326:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.243311:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ced500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660235264:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.243347:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287aca28 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8316 expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.243374:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287aca28 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8316 expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.243388:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0000, rc 0) 00000100:00100000:1.0:1788845028.243537:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641511040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2423us (2652us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.243546:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 586 00000100:00100000:0.0:1788845028.249168:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.249272:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641511168 00000100:00100000:2.0:1788845028.249281:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641511168 00000100:00100000:2.0:1788845028.249337:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 587 00000100:00100000:2.0:1788845028.249347:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641511168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.249365:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.249383:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca2f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a831d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.249442:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660235392:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.249634:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.252584:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660235392:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.252636:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca2f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a831d expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.252661:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca2f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a831d expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.252674:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:2.0:1788845028.252732:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641511168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3387us (3574us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.252741:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 587 00000100:00100000:0.0:1788845028.255289:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.255357:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641511296 00000100:00100000:3.0:1788845028.255367:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641511296 00000100:00100000:3.0:1788845028.255426:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 588 00000100:00100000:3.0:1788845028.255437:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641511296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.255467:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.255487:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca36 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8324 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.255545:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660235520:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845028.255634:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.257936:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660235520:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845028.257974:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca36 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8324 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.257999:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca36 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8324 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.258027:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961800, rc 0) 00000100:00100000:3.0:1788845028.258119:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641511296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2684us (2836us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845028.258128:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 588 00000100:00100000:0.0:1788845028.265159:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.265548:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641511424 00000100:00100000:3.0:1788845028.265558:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641511424 00000100:00100000:3.0:1788845028.265623:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 589 00000100:00100000:3.0:1788845028.265631:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641511424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.265649:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.265666:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca3d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a832b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.265722:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660235648:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845028.266039:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.270150:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660235648:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845028.270189:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca3d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a832b expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.270215:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca3d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a832b expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.270228:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961800, rc 0) 00000100:00100000:3.0:1788845028.270388:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641511424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4757us (5239us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845028.270410:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 589 00000100:00100000:0.0:1788845028.274097:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.274323:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641511552 00000100:00100000:2.0:1788845028.274335:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641511552 00000100:00100000:2.0:1788845028.274421:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 590 00000100:00100000:2.0:1788845028.274431:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641511552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.274464:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.274484:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8332 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.274518:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.274528:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 5) 00010000:00010000:2.0:1788845028.274532:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:2.0:1788845028.274537:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1597440->1601535] (req 1597440->1601535) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.274567:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.274584:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 10 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.274598:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660235776:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845028.274683:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.278312:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.278394:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641511680 00000100:00100000:3.0:1788845028.278403:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641511680 00010000:00010000:3.0:1788845028.278455:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 11 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.278500:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 322 00000100:00100000:3.0:1788845028.278510:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641511680:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845028.278525:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845028.278549:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 11 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.278567:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 11 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.278592:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:3.0:1788845028.278596:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:3.0:1788845028.278605:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.278627:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.278666:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.278704:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660235904:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845028.278789:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00010000:00010000:2.0:1788845028.279349:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ff7e868a/0xb2187301287aca13 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1355776->1359871) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8301 expref: 11 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.279386:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660235776:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845028.279412:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.279423:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:2.0:1788845028.279483:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641511552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5053us (5396us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.279492:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 590 00000100:00100000:0.0:1788845028.284919:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.285073:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641511808 00000100:00100000:2.0:1788845028.285084:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641511808 00002000:00100000:2.0:1788845028.285134:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1601535) req@ffff94a0359ae300 x1875739641511808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.285145:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359ae300 with x1875739641511808 ext(1597440->1601535) 00002000:00010000:2.0:1788845028.285151:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359ae300 00000100:00100000:2.0:1788845028.285178:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 341 00000100:00100000:2.0:1788845028.285187:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641511808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.285433:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641511808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 246us (559us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.285456:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 341 00002000:00100000:2.0:1788845028.285464:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1601535) req@ffff94a0359ae300 x1875739641511808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.285472:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359ae300 with x1875739641511808 ext(1597440->1601535) 00002000:00010000:2.0:1788845028.285490:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359ae300 00000100:00100000:3.0:1788845028.285776:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660235904:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845028.285810:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845028.285848:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641511680:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7340us (7546us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.285856:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 322 00000100:00100000:0.0:1788845028.289794:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.289937:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641511936 00000100:00100000:1.0:1788845028.289947:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641511936 00002000:00100000:1.0:1788845028.289996:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1601536->1605631) req@ffff94a0359ad500 x1875739641511936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.290006:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359ad500 with x1875739641511936 ext(1601536->1605631) 00002000:00010000:1.0:1788845028.290011:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359ad500 00000100:00100000:1.0:1788845028.290056:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 342 00000100:00100000:1.0:1788845028.290063:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641511936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.290285:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641511936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 222us (500us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845028.290292:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 342 00002000:00100000:1.0:1788845028.290298:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1601536->1605631) req@ffff94a0359ad500 x1875739641511936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.290305:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359ad500 with x1875739641511936 ext(1601536->1605631) 00002000:00010000:1.0:1788845028.290308:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359ad500 00000100:00100000:2.0:1788845028.295037:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.295138:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641512064 00000100:00100000:1.0:1788845028.295148:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641512064 00002000:00100000:1.0:1788845028.295202:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1605632->1609727) req@ffff949f01cec380 x1875739641512064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.295214:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cec380 with x1875739641512064 ext(1605632->1609727) 00002000:00010000:1.0:1788845028.295220:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cec380 00000100:00100000:1.0:1788845028.295257:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 343 00000100:00100000:1.0:1788845028.295267:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641512064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.295528:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641512064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 262us (499us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845028.295537:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 343 00002000:00100000:1.0:1788845028.295544:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1605632->1609727) req@ffff949f01cec380 x1875739641512064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.295553:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cec380 with x1875739641512064 ext(1605632->1609727) 00002000:00010000:1.0:1788845028.295564:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cec380 00000100:00100000:0.0:1788845028.299652:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.299820:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641512192 00000100:00100000:2.0:1788845028.299831:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641512192 00000100:00100000:2.0:1788845028.299911:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 591 00000100:00100000:2.0:1788845028.299920:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641512192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.299949:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.299971:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8339 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.299998:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:2.0:1788845028.300002:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:2.0:1788845028.300013:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.300030:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.300044:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0600, rc 0) 00000100:00100000:2.0:1788845028.300100:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641512192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 180us (457us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.300106:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 591 00000100:00100000:2.0:1788845028.304942:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.305111:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641512320 00000100:00100000:3.0:1788845028.305120:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641512320 00000100:00100000:3.0:1788845028.305174:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 592 00000100:00100000:3.0:1788845028.305184:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641512320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.305205:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.305226:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8340 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.305262:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.305272:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.305279:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:3.0:1788845028.305283:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:3.0:1788845028.305289:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1155072->1187839] (req 1155072->1187839) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.305350:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.305369:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 10 pid: 6684 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.305383:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660236032:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845028.305615:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.305625:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 13 pid: 6685 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.305633:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03456a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660236160:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845028.305655:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.309838:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660236032:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845028.309859:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03456a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660236160:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845028.309938:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1155072->1187839] (req 1155072->1187839) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.309962:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961800, rc 0) 00000100:00100000:3.0:1788845028.310031:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641512320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4848us (5107us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.310054:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 592 00000100:00100000:0.0:1788845028.313019:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.313050:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.313134:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641512576 00000100:00100000:1.0:1788845028.313143:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641512576 00010000:00010000:1.0:1788845028.313180:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 13 pid: 6685 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.313211:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641512448 00000100:00100000:1.0:1788845028.313214:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641512448 00010000:00010000:1.0:1788845028.313224:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 11 pid: 6684 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.313246:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 323 00000100:00100000:1.0:1788845028.313262:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641512576:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845028.313276:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845028.313299:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 13 pid: 6685 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.313311:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 13 pid: 6685 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.313333:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287aca44 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1597440->1601535) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8332 expref: 13 pid: 6685 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.313349:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845028.313394:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641512576:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 141us (514us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.313401:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 323 00000100:00100000:1.0:1788845028.313428:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 324 00000100:00100000:1.0:1788845028.313434:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641512448:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845028.313440:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845028.313462:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 11 pid: 6684 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.313475:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 11 pid: 6684 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.313486:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287aca4b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8339 expref: 11 pid: 6684 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.313495:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:1.0:1788845028.313499:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:1.0:1788845028.313506:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.313514:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.313547:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.313559:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660236288:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845028.313726:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.317878:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.318277:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641512704 00000100:00100000:3.0:1788845028.318286:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641512704 00002000:00100000:3.0:1788845028.318337:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1155072->1159167) req@ffff949f01cee680 x1875739641512704/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.318348:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cee680 with x1875739641512704 ext(1155072->1159167) 00002000:00010000:3.0:1788845028.318354:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cee680 00000100:00100000:3.0:1788845028.318387:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 344 00000100:00100000:3.0:1788845028.318397:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641512704:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.319378:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660236288:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845028.319406:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845028.319767:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641512704:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1370us (1896us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845028.319776:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 344 00002000:00100000:3.0:1788845028.319788:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1155072->1159167) req@ffff949f01cee680 x1875739641512704/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.319808:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cee680 with x1875739641512704 ext(1155072->1159167) 00002000:00010000:3.0:1788845028.319814:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cee680 00000100:00100000:1.0:1788845028.321125:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641512448:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7682us (8064us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.321138:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 324 00000100:00100000:2.0:1788845028.325486:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.325593:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641512832 00000100:00100000:1.0:1788845028.325602:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641512832 00002000:00100000:1.0:1788845028.325657:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1183744->1187839) req@ffff949f01cedc00 x1875739641512832/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.325669:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cedc00 with x1875739641512832 ext(1183744->1187839) 00002000:00010000:1.0:1788845028.325675:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cedc00 00000100:00100000:1.0:1788845028.325704:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 345 00000100:00100000:1.0:1788845028.325713:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641512832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.325973:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641512832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 260us (495us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845028.325981:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 345 00002000:00100000:1.0:1788845028.325988:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1183744->1187839) req@ffff949f01cedc00 x1875739641512832/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.325995:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cedc00 with x1875739641512832 ext(1183744->1187839) 00002000:00010000:1.0:1788845028.325999:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cedc00 00000100:00100000:1.0:1788845028.330394:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.330509:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641512960 00000100:00100000:2.0:1788845028.330518:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641512960 00000100:00100000:2.0:1788845028.330588:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 593 00000100:00100000:2.0:1788845028.330599:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641512960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.331022:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.331041:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287aca59 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8347 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.331102:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660236416:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.331175:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.334407:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660236416:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.334467:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287aca59 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8347 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.334496:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287aca59 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8347 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.334508:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0000, rc 0) 00000100:00100000:2.0:1788845028.334573:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641512960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3975us (4187us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.334582:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 593 00000100:00100000:0.0:1788845028.338538:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.338636:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641513088 00000100:00100000:2.0:1788845028.338646:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641513088 00000100:00100000:2.0:1788845028.338711:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 594 00000100:00100000:2.0:1788845028.338721:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641513088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.338741:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.338760:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca60 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a834e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.338859:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660236544:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.338991:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.341677:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660236544:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.341721:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca60 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a834e expref: 11 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.341748:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca60 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a834e expref: 11 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.341763:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:2.0:1788845028.341836:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641513088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3117us (3308us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.341845:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 594 00000100:00100000:1.0:1788845028.346627:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.346767:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641513216 00000100:00100000:3.0:1788845028.346777:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641513216 00000100:00100000:3.0:1788845028.346842:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 595 00000100:00100000:3.0:1788845028.346853:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641513216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.346873:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.346917:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e5bbb4af/0xb2187301287aca67 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8355 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.347013:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03456a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660236672:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845028.347096:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.350359:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03456a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660236672:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845028.350404:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e5bbb4af/0xb2187301287aca67 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8355 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.350431:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e5bbb4af/0xb2187301287aca67 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8355 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.350455:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960e00, rc 0) 00000100:00100000:3.0:1788845028.350518:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641513216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3667us (3910us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845028.350527:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 595 00000100:00100000:0.0:1788845028.357031:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.357204:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641513344 00000100:00100000:3.0:1788845028.357216:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641513344 00000100:00100000:3.0:1788845028.357286:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 596 00000100:00100000:3.0:1788845028.357298:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641513344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.357317:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.357334:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e5bbb4af/0xb2187301287aca6e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a835c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.357391:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03456a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660236800:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845028.358217:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.368363:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03456a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660236800:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845028.368407:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e5bbb4af/0xb2187301287aca6e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a835c expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.368432:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e5bbb4af/0xb2187301287aca6e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a835c expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.368747:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960e00, rc 0) 00000100:00100000:3.0:1788845028.369093:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641513344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11794us (12069us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845028.369104:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 596 00000100:00100000:1.0:1788845028.379760:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.379934:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641513472 00000100:00100000:0.0:1788845028.379944:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641513472 00000100:00100000:0.0:1788845028.380003:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 886 00000100:00100000:0.0:1788845028.380015:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641513472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845028.380039:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845028.380043:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788845028.380089:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287ac9fe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a82ec expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.380109:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.380114:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845028.380128:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8363 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.380157:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.380167:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.380223:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.380242:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 10 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.380272:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660236928:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845028.380349:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.386640:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.386756:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660236928:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845028.386793:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.386827:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.386831:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.386836:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.386847:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f028c2e00, rc 0) 00000100:00100000:1.0:1788845028.386961:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641513472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6948us (7210us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.386970:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 886 00000100:00100000:2.0:1788845028.388136:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641513600 00000100:00100000:2.0:1788845028.388148:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641513600 00010000:00010000:2.0:1788845028.388187:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 11 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.388230:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 325 00000100:00100000:2.0:1788845028.388241:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641513600:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845028.388269:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845028.388297:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 11 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.388333:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 11 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.388354:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca21 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8308 expref: 11 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.388372:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.388379:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.388391:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.388400:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.388439:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.388638:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660237056:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845028.388818:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.395287:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.395398:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641513728 00000100:00100000:3.0:1788845028.395407:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641513728 00002000:00100000:3.0:1788845028.395470:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035689880 x1875739641513728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.395482:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035689880 with x1875739641513728 00000100:00100000:3.0:1788845028.395525:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 559 00000100:00100000:3.0:1788845028.395535:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641513728:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845028.395962:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641513728:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 427us (683us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845028.395971:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 559 00002000:00100000:3.0:1788845028.395978:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035689880 x1875739641513728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.395986:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035689880 with x1875739641513728 00000100:00100000:2.0:1788845028.397989:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660237056:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845028.398014:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845028.398068:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641513600:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9828us (11437us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.398094:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 325 00000100:00100000:1.0:1788845028.400398:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.400503:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641513856 00000100:00100000:0.0:1788845028.400513:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641513856 00002000:00100000:0.0:1788845028.400640:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035688000 x1875739641513856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.400652:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035688000 with x1875739641513856 00000100:00100000:0.0:1788845028.400693:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 560 00000100:00100000:0.0:1788845028.400703:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641513856:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845028.401026:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641513856:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 323us (636us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845028.401034:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 560 00002000:00100000:0.0:1788845028.401040:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035688000 x1875739641513856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.401047:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035688000 with x1875739641513856 00000100:00100000:3.0:1788845028.406494:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.406615:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641513984 00000100:00100000:1.0:1788845028.406628:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641513984 00000100:00100000:1.0:1788845028.406703:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 887 00000100:00100000:1.0:1788845028.406715:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641513984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.406738:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.406761:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aca7c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a836a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.406800:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.406846:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660237184:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.406978:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000004:00010000:2.0:1788845028.411753:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845028 00000100:00100000:2.0:1788845028.411782:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660237184:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845028.411867:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845028.411973:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aca7c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a836a expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.411994:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.411998:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845028.412002:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aca7c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a836a expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.412017:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756000, rc 0) 00000100:00100000:2.0:1788845028.412123:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641513984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5410us (5639us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.412132:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 887 00000100:00100000:3.0:1788845028.416408:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.416587:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641514112 00000100:00100000:2.0:1788845028.416598:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641514112 00000100:00100000:2.0:1788845028.416664:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 888 00000100:00100000:2.0:1788845028.416674:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641514112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.416695:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.416715:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287aca83 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8371 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.416756:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.416832:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660237312:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.416962:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.422961:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660237312:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845028.422990:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845028.423009:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287aca83 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8371 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.423029:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.423032:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845028.423037:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287aca83 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8371 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.423052:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:2.0:1788845028.423117:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641514112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6443us (6717us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.423126:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 888 00000100:00100000:0.0:1788845028.434411:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.434767:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641514240 00000100:00100000:2.0:1788845028.434777:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641514240 00000100:00100000:2.0:1788845028.435108:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 597 00000100:00100000:2.0:1788845028.435120:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641514240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.435142:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.435160:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8378 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.435193:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.435219:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 5) 00010000:00010000:2.0:1788845028.435224:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:2.0:1788845028.435229:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1523712->1531903] (req 1523712->1531903) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.435279:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.435298:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 11 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.435313:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660237440:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845028.435396:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.439383:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.439487:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641514368 00000100:00100000:0.0:1788845028.439509:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641514368 00002000:00100000:0.0:1788845028.439568:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1155072->1187839) req@ffff94a03f6c4000 x1875739641514368/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.439579:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c4000 with x1875739641514368 ext(1155072->1187839) 00010000:00010000:0.0:1788845028.439589:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 12 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.439609:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 12 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845028.439617:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f6c4000 00000100:00100000:0.0:1788845028.439649:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 15 00000100:00100000:0.0:1788845028.439658:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641514368:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845028.440572:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660237440:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845028.440605:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1523712->1531903] (req 1523712->1531903) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.440622:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:2.0:1788845028.440668:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:14891:x1875739641514240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5551us (6266us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.440677:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 597 00000100:00100000:0.0:1788845028.445662:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641514368:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5993us (6277us total) trans 4294967408 rc 0/0 00000100:00100000:0.0:1788845028.445677:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 15 00002000:00100000:0.0:1788845028.445704:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1155072->1187839) req@ffff94a03f6c4000 x1875739641514368/t4294967408(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/448 e 0 to 0 dl 1788845039 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.445715:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c4000 with x1875739641514368 ext(1155072->1187839) 00010000:00010000:0.0:1788845028.445727:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 14 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.445750:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 14 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845028.445779:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f6c4000 00000100:00100000:0.0:1788845028.451320:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.452752:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641514496 00000100:00100000:2.0:1788845028.452764:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641514496 00010000:00010000:2.0:1788845028.452865:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 14 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.453007:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 326 00000100:00100000:2.0:1788845028.453019:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641514496:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845028.453036:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845028.453059:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 14 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.453077:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 14 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.453102:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa09650e/0xb2187301287aca52 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1155072->1187839) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8340 expref: 14 pid: 6682 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.453119:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:2.0:1788845028.453123:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:2.0:1788845028.453132:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.453142:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.453182:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.453196:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660237568:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845028.453357:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.463319:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660237568:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845028.463352:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845028.463420:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641514496:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10403us (12171us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.463432:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 326 00000100:00100000:0.0:1788845028.466988:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.467031:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.467141:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641514752 00000100:00100000:2.0:1788845028.467152:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641514752 00002000:00100000:2.0:1788845028.467199:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be4700 x1875739641514752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 504/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.467212:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be4700 with x1875739641514752 00000100:00100000:2.0:1788845028.467252:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 561 00000100:00100000:2.0:1788845028.467261:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641514752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.468469:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641514624 00000100:00100000:1.0:1788845028.468492:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641514624 00002000:00100000:1.0:1788845028.468535:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1896447) req@ffff94a035be6d80 x1875739641514624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.468559:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be6d80 with x1875739641514624 ext(1048576->1896447) 00002000:00010000:1.0:1788845028.468565:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be6d80 00000100:00100000:1.0:1788845028.468594:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 346 00000100:00100000:1.0:1788845028.468603:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641514624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.502611:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641514752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 35335us (35619us total) trans 0 rc 987136/987136 00000100:00100000:1.0:1788845028.502626:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 561 00002000:00100000:1.0:1788845028.502640:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be4700 x1875739641514752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 504/440 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 987136/987136 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.502649:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be4700 with x1875739641514752 00000100:00100000:1.0:1788845028.505038:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641514624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 36425us (37994us total) trans 0 rc 847872/847872 00000100:00100000:1.0:1788845028.505053:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 346 00002000:00100000:1.0:1788845028.505064:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1896447) req@ffff94a035be6d80 x1875739641514624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/440 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 847872/847872 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.505078:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be6d80 with x1875739641514624 ext(1048576->1896447) 00002000:00010000:1.0:1788845028.505085:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be6d80 00000100:00100000:0.0:1788845028.514279:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.514347:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.514547:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641514880 00000100:00100000:2.0:1788845028.514558:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641514880 00000100:00100000:2.0:1788845028.514635:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 889 00000100:00100000:2.0:1788845028.514647:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641514880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.514671:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.514696:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287aca91 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a837f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.514739:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.514782:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660237696:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.514932:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.515069:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641515008 00000100:00100000:1.0:1788845028.515074:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641515008 00000100:00100000:1.0:1788845028.515146:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 598 00000100:00100000:1.0:1788845028.515154:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641515008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.515173:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.515184:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8386 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.515204:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:1.0:1788845028.515208:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:1.0:1788845028.515216:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.515228:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.515238:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757400, rc 0) 00000100:00100000:1.0:1788845028.515261:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641515008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 110us (915us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.515266:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 598 00000100:00100000:2.0:1788845028.517320:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660237696:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845028.517351:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845028.517372:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287aca91 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a837f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.517392:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.517395:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845028.517400:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287aca91 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a837f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.517414:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0000, rc 0) 00000100:00100000:2.0:1788845028.517566:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641514880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2921us (3297us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.517572:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 889 00000100:00100000:0.0:1788845028.521274:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.521674:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641515136 00000100:00100000:1.0:1788845028.521685:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641515136 00000100:00100000:1.0:1788845028.521746:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 890 00000100:00100000:1.0:1788845028.521756:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641515136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.521777:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.521808:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca9f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a838d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.521847:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.521935:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660237824:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.522057:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.525801:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660237824:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.525833:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845028.525855:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca9f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a838d expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.525875:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.525877:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.525994:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aca9f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a838d expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.526013:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757800, rc 0) 00000100:00100000:1.0:1788845028.526069:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641515136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4313us (4806us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.526079:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 890 00000100:00100000:0.0:1788845028.530087:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.530170:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641515264 00000100:00100000:3.0:1788845028.530182:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641515264 00000100:00100000:3.0:1788845028.530240:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 599 00000100:00100000:3.0:1788845028.530248:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:14891:x1875739641515264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.530266:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.530283:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8394 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.530310:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.530317:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.530324:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:3.0:1788845028.530328:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:3.0:1788845028.530334:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1200128->1249279] (req 1200128->1249279) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 12 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.530375:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.530394:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 13 pid: 6685 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.530431:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660237952:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845028.530531:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.530542:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.530551:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660238080:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845028.530575:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.537156:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.537251:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.537257:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660237952:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845028.537272:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660238080:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845028.537306:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1200128->1249279] (req 1200128->1249279) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 12 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.537329:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961000, rc 0) 00000100:00100000:3.0:1788845028.537392:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+12:14891:x1875739641515264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7144us (7312us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.537401:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 599 00000100:00100000:2.0:1788845028.537849:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641515392 00000100:00100000:2.0:1788845028.537859:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641515392 00010000:00010000:2.0:1788845028.538465:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 14 pid: 6685 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.538603:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641515520 00000100:00100000:0.0:1788845028.538609:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641515520 00010000:00010000:0.0:1788845028.538631:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.539142:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 327 00000100:00100000:1.0:1788845028.539161:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008916450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641515392:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845028.539180:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845028.539206:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 14 pid: 6685 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.539226:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 14 pid: 6685 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.539280:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aca98 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8386 expref: 14 pid: 6685 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.539301:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845028.539795:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 328 00000100:00100000:3.0:1788845028.539807:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641515520:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845028.539818:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845028.539839:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.539853:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.539869:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287aca8a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1523712->1531903) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8378 expref: 12 pid: 6684 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.539880:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:3.0:1788845028.539910:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:3.0:1788845028.539918:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 11 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.539937:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 11 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.539953:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008916450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641515392:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 800us (2806us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.539960:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 327 00010000:00010000:3.0:1788845028.539973:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 11 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.539986:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660238208:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845028.540038:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.543058:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660238208:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845028.543085:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845028.543136:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641515520:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3331us (5883us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.543558:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 328 00000100:00100000:2.0:1788845028.549418:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.549596:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641515648 00000100:00100000:0.0:1788845028.549666:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641515648 00002000:00100000:0.0:1788845028.549719:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1896447) req@ffff949f01cec000 x1875739641515648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.549731:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cec000 with x1875739641515648 ext(1048576->1896447) 00002000:00010000:0.0:1788845028.549737:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cec000 00000100:00100000:0.0:1788845028.549766:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 347 00000100:00100000:0.0:1788845028.549776:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641515648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.565615:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641515648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 15821us (16186us total) trans 0 rc 847872/847872 00000100:00100000:1.0:1788845028.565627:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 347 00002000:00100000:1.0:1788845028.565640:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1896447) req@ffff949f01cec000 x1875739641515648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/440 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 847872/847872 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.565648:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cec000 with x1875739641515648 ext(1048576->1896447) 00002000:00010000:1.0:1788845028.565653:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cec000 00000100:00100000:3.0:1788845028.572249:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.572300:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.572429:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641515776 00000100:00100000:2.0:1788845028.572446:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641515776 00000100:00100000:2.0:1788845028.572511:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 891 00000100:00100000:2.0:1788845028.572521:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641515776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.572544:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.572565:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287acaad lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a839b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.572629:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.572672:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660238336:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.572770:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.574157:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641515904 00000100:00100000:0.0:1788845028.574166:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641515904 00000100:00100000:0.0:1788845028.574237:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 600 00000100:00100000:0.0:1788845028.574247:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641515904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845028.574264:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845028.574280:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287acab4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83a2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.574328:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660238464:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845028.574399:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.577678:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660238464:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845028.577723:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287acab4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83a2 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.577749:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287acab4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83a2 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.577763:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:0.0:1788845028.577872:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641515904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3625us (5569us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845028.577932:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 600 00000100:00100000:3.0:1788845028.580057:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660238336:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845028.580086:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845028.580106:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287acaad lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a839b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.580126:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845028.580129:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845028.580134:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287acaad lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a839b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.580147:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1000, rc 0) 00000100:00100000:2.0:1788845028.580267:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641515776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7745us (8027us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.580277:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 891 00000100:00100000:3.0:1788845028.584178:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.586975:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641516032 00000100:00100000:2.0:1788845028.586991:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641516032 00000100:00100000:2.0:1788845028.587096:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 892 00000100:00100000:2.0:1788845028.587107:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641516032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.587128:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.587159:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acabb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83a9 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.587197:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.587241:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660238592:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.587642:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.590535:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.590721:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641516160 00000100:00100000:2.0:1788845028.590731:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641516160 00000100:00100000:1.0:1788845028.591141:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660238592:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.591166:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845028.591186:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acabb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83a9 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.591211:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.591215:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.591221:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acabb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83a9 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.591268:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:1.0:1788845028.591336:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641516032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4230us (7167us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.591345:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 892 00000100:00100000:2.0:1788845028.593095:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 601 00000100:00100000:2.0:1788845028.593121:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641516160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.593155:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.593178:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287acac2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83b0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.593241:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660238720:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.593334:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.602498:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660238720:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.602542:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287acac2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83b0 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.602569:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000046200bc4/0xb2187301287acac2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83b0 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.602586:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845028.602751:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641516160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9641us (12223us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.602761:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 601 00000100:00100000:0.0:1788845028.608085:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.608160:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.608187:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641516416 00000100:00100000:0.0:1788845028.608197:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641516416 00000100:00100000:3.0:1788845028.608247:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641516288 00000100:00100000:3.0:1788845028.608251:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641516288 00000100:00100000:2.0:1788845028.608304:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 893 00000100:00100000:2.0:1788845028.608317:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641516288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.608342:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.608364:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acac9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83b7 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.608406:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.608469:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660238848:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845028.608836:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 602 00000100:00100000:0.0:1788845028.608847:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641516416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845028.608863:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845028.608874:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287acad0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83be expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.608937:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023ddc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660238976:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.609267:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.609670:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.612484:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023ddc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660238976:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.612534:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287acad0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83be expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.612563:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287acad0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83be expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.612577:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f400, rc 0) 00000100:00100000:1.0:1788845028.612693:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641516416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3848us (4531us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.612703:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 602 00000100:00100000:0.0:1788845028.612818:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ceed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660238848:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845028.612835:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:0.0:1788845028.612848:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acac9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83b7 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.612863:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.612866:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845028.612871:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acac9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83b7 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.612909:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:0.0:1788845028.612947:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641516288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4634us (4870us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845028.612955:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 893 00000100:00100000:1.0:1788845028.618117:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.618428:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641516544 00000100:00100000:3.0:1788845028.618448:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641516544 00000100:00100000:3.0:1788845028.618552:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 603 00000100:00100000:3.0:1788845028.618563:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641516544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.618584:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.618613:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83c5 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.618648:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.618678:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 5) 00010000:00010000:3.0:1788845028.618683:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:3.0:1788845028.618689:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1777664->1781759] (req 1777664->1781759) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 14 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.618746:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.618765:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 10 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.618779:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660239104:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845028.619235:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.622279:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.622383:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660239104:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845028.622417:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1777664->1781759] (req 1777664->1781759) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 14 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.622445:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960600, rc 0) 00000100:00100000:1.0:1788845028.622480:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641516672 00000100:00100000:1.0:1788845028.622487:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641516672 00000100:00100000:3.0:1788845028.622512:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641516544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3952us (4405us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.622520:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 603 00002000:00100000:1.0:1788845028.622530:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1200128->1249279) req@ffff949f01ceed80 x1875739641516672/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.622539:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01ceed80 with x1875739641516672 ext(1200128->1249279) 00010000:00010000:1.0:1788845028.622546:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 11 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.622590:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 11 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845028.622598:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f01ceed80 00000100:00100000:1.0:1788845028.622638:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 16 00000100:00100000:1.0:1788845028.622646:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641516672:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845028.631406:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641516672:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 8750us (9128us total) trans 4294967409 rc 0/0 00000100:00100000:1.0:1788845028.631420:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 16 00002000:00100000:1.0:1788845028.631432:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1200128->1249279) req@ffff949f01ceed80 x1875739641516672/t4294967409(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/448 e 0 to 0 dl 1788845039 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.631452:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01ceed80 with x1875739641516672 ext(1200128->1249279) 00010000:00010000:1.0:1788845028.631462:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 13 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.631485:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 13 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845028.631493:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f01ceed80 00000100:00100000:2.0:1788845028.637003:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.637401:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641516800 00000100:00100000:1.0:1788845028.637412:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641516800 00010000:00010000:1.0:1788845028.637461:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 13 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.637528:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 329 00000100:00100000:1.0:1788845028.637539:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641516800:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845028.637557:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845028.637581:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 13 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.637598:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 13 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.637621:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d16ff963/0xb2187301287acaa6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1200128->1249279) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8394 expref: 13 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.637636:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:1.0:1788845028.637641:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:1.0:1788845028.637649:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.637659:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.637697:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.637712:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359aea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660239232:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845028.637859:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.642653:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.642740:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641516928 00000100:00100000:2.0:1788845028.642750:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641516928 00002000:00100000:2.0:1788845028.642809:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1777664->1781759) req@ffff949f023dea00 x1875739641516928/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.642820:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dea00 with x1875739641516928 ext(1777664->1781759) 00002000:00010000:2.0:1788845028.642827:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dea00 00000100:00100000:2.0:1788845028.642931:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 348 00000100:00100000:2.0:1788845028.642943:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641516928:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.643182:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641516928:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 240us (540us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.643191:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 348 00002000:00100000:2.0:1788845028.643199:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1777664->1781759) req@ffff949f023dea00 x1875739641516928/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.643207:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dea00 with x1875739641516928 ext(1777664->1781759) 00002000:00010000:2.0:1788845028.643211:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dea00 00000100:00100000:1.0:1788845028.646693:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359aea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660239232:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845028.646724:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845028.646810:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641516800:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9272us (9817us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.646822:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 329 00000100:00100000:1.0:1788845028.647639:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.647727:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641517056 00000100:00100000:1.0:1788845028.647735:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641517056 00002000:00100000:1.0:1788845028.647777:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1781760->1785855) req@ffff94a0359aea00 x1875739641517056/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.647787:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359aea00 with x1875739641517056 ext(1781760->1785855) 00002000:00010000:1.0:1788845028.647793:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359aea00 00000100:00100000:1.0:1788845028.647821:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 349 00000100:00100000:1.0:1788845028.647831:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641517056:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.648079:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641517056:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (442us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845028.648087:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 349 00002000:00100000:1.0:1788845028.648093:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1781760->1785855) req@ffff94a0359aea00 x1875739641517056/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.648101:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359aea00 with x1875739641517056 ext(1781760->1785855) 00002000:00010000:1.0:1788845028.648104:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359aea00 00000100:00100000:1.0:1788845028.655047:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.660067:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641517184 00000100:00100000:2.0:1788845028.660084:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641517184 00002000:00100000:2.0:1788845028.660147:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1785856->1789951) req@ffff94a0359adc00 x1875739641517184/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.660181:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359adc00 with x1875739641517184 ext(1785856->1789951) 00002000:00010000:2.0:1788845028.660187:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359adc00 00000100:00100000:2.0:1788845028.660224:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 350 00000100:00100000:2.0:1788845028.660235:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641517184:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.660655:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641517184:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 420us (5616us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.660664:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 350 00002000:00100000:2.0:1788845028.660672:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1785856->1789951) req@ffff94a0359adc00 x1875739641517184/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.660680:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359adc00 with x1875739641517184 ext(1785856->1789951) 00002000:00010000:2.0:1788845028.660685:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359adc00 00000100:00100000:1.0:1788845028.665056:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.665375:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641517312 00000100:00100000:3.0:1788845028.665389:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641517312 00002000:00100000:3.0:1788845028.665456:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1789952->1794047) req@ffff94a035be4000 x1875739641517312/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.665474:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be4000 with x1875739641517312 ext(1789952->1794047) 00002000:00010000:3.0:1788845028.665481:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be4000 00000100:00100000:3.0:1788845028.665518:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 351 00000100:00100000:3.0:1788845028.665529:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641517312:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845028.665776:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641517312:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (728us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845028.665784:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 351 00002000:00100000:3.0:1788845028.665791:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1789952->1794047) req@ffff94a035be4000 x1875739641517312/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.665812:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be4000 with x1875739641517312 ext(1789952->1794047) 00002000:00010000:3.0:1788845028.665816:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be4000 00000100:00100000:0.0:1788845028.669531:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.669671:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641517440 00000100:00100000:2.0:1788845028.669682:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641517440 00002000:00100000:2.0:1788845028.669731:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1794048->1798143) req@ffff949f023df480 x1875739641517440/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.669742:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023df480 with x1875739641517440 ext(1794048->1798143) 00002000:00010000:2.0:1788845028.669748:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023df480 00000100:00100000:2.0:1788845028.669780:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 352 00000100:00100000:2.0:1788845028.669791:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641517440:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.670139:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641517440:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 349us (617us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.670147:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 352 00002000:00100000:2.0:1788845028.670169:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1794048->1798143) req@ffff949f023df480 x1875739641517440/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.670178:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023df480 with x1875739641517440 ext(1794048->1798143) 00002000:00010000:2.0:1788845028.670182:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023df480 00000100:00100000:0.0:1788845028.673871:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.674194:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641517568 00000100:00100000:1.0:1788845028.674205:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641517568 00002000:00100000:1.0:1788845028.674260:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1798144->1802239) req@ffff949f023df800 x1875739641517568/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.674272:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023df800 with x1875739641517568 ext(1798144->1802239) 00002000:00010000:1.0:1788845028.674278:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023df800 00000100:00100000:1.0:1788845028.674313:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 353 00000100:00100000:1.0:1788845028.674324:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641517568:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.674581:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641517568:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 258us (719us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845028.674589:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 353 00002000:00100000:1.0:1788845028.674597:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1798144->1802239) req@ffff949f023df800 x1875739641517568/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.674604:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023df800 with x1875739641517568 ext(1798144->1802239) 00002000:00010000:1.0:1788845028.674607:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023df800 00000100:00100000:0.0:1788845028.678076:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.678172:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641517696 00000100:00100000:3.0:1788845028.678185:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641517696 00002000:00100000:3.0:1788845028.678245:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1802240->1806335) req@ffff949f023ddf80 x1875739641517696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.678257:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023ddf80 with x1875739641517696 ext(1802240->1806335) 00002000:00010000:3.0:1788845028.678264:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023ddf80 00000100:00100000:3.0:1788845028.678298:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 354 00000100:00100000:3.0:1788845028.678308:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641517696:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845028.678557:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641517696:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 249us (490us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845028.678565:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 354 00002000:00100000:3.0:1788845028.678589:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1802240->1806335) req@ffff949f023ddf80 x1875739641517696/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.678597:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023ddf80 with x1875739641517696 ext(1802240->1806335) 00002000:00010000:3.0:1788845028.678600:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023ddf80 00000100:00100000:0.0:1788845028.681744:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.681939:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641517824 00000100:00100000:2.0:1788845028.681949:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641517824 00002000:00100000:2.0:1788845028.682004:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1806336->1810431) req@ffff949f023dc700 x1875739641517824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.682015:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dc700 with x1875739641517824 ext(1806336->1810431) 00002000:00010000:2.0:1788845028.682021:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dc700 00000100:00100000:2.0:1788845028.682046:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 355 00000100:00100000:2.0:1788845028.682056:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641517824:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.682313:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641517824:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 257us (579us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.682321:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 355 00002000:00100000:2.0:1788845028.682328:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1806336->1810431) req@ffff949f023dc700 x1875739641517824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.682335:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dc700 with x1875739641517824 ext(1806336->1810431) 00002000:00010000:2.0:1788845028.682338:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dc700 00000100:00100000:0.0:1788845028.686170:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.686292:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641517952 00000100:00100000:1.0:1788845028.686302:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641517952 00002000:00100000:1.0:1788845028.686352:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1810432->1814527) req@ffff949f023de680 x1875739641517952/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.686363:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023de680 with x1875739641517952 ext(1810432->1814527) 00002000:00010000:1.0:1788845028.686369:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023de680 00000100:00100000:1.0:1788845028.686404:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 356 00000100:00100000:1.0:1788845028.686414:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641517952:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845028.686664:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641517952:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 251us (504us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845028.686679:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 356 00002000:00100000:1.0:1788845028.686687:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1810432->1814527) req@ffff949f023de680 x1875739641517952/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845028.686694:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023de680 with x1875739641517952 ext(1810432->1814527) 00002000:00010000:1.0:1788845028.686697:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023de680 00000100:00100000:0.0:1788845028.690490:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.690781:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641518080 00000100:00100000:3.0:1788845028.690851:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641518080 00002000:00100000:3.0:1788845028.690993:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1818623) req@ffff949f023dca80 x1875739641518080/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.691004:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dca80 with x1875739641518080 ext(1814528->1818623) 00002000:00010000:3.0:1788845028.691009:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dca80 00000100:00100000:3.0:1788845028.691094:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 357 00000100:00100000:3.0:1788845028.691105:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641518080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845028.691372:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641518080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (894us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845028.691381:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 357 00002000:00100000:3.0:1788845028.691388:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1818623) req@ffff949f023dca80 x1875739641518080/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845028.691396:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dca80 with x1875739641518080 ext(1814528->1818623) 00002000:00010000:3.0:1788845028.691400:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dca80 00000100:00100000:0.0:1788845028.694846:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.695007:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641518208 00000100:00100000:2.0:1788845028.695017:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641518208 00002000:00100000:2.0:1788845028.695072:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1818624->1822719) req@ffff949f023ddc00 x1875739641518208/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.695085:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023ddc00 with x1875739641518208 ext(1818624->1822719) 00002000:00010000:2.0:1788845028.695105:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023ddc00 00000100:00100000:2.0:1788845028.695140:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 358 00000100:00100000:2.0:1788845028.695150:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641518208:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845028.695593:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641518208:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 442us (756us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845028.695603:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 358 00002000:00100000:2.0:1788845028.695610:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1818624->1822719) req@ffff949f023ddc00 x1875739641518208/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:134/0 lens 488/4536 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.695618:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023ddc00 with x1875739641518208 ext(1818624->1822719) 00002000:00010000:2.0:1788845028.695622:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023ddc00 00000100:00100000:0.0:1788845028.701844:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.702031:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641518336 00000100:00100000:1.0:1788845028.702042:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641518336 00000100:00100000:1.0:1788845028.702107:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 604 00000100:00100000:1.0:1788845028.702116:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641518336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.702135:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.702155:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83cc expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.702179:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:1.0:1788845028.702183:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:1.0:1788845028.702193:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.702211:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.702224:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757c00, rc 0) 00000100:00100000:1.0:1788845028.702290:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2368:x1875739641518336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 175us (459us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.702297:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 604 00000100:00100000:0.0:1788845028.706659:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.706757:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641518464 00000100:00100000:1.0:1788845028.706768:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641518464 00000100:00100000:1.0:1788845028.706847:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 894 00000100:00100000:1.0:1788845028.706857:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641518464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.706879:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.706944:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acae5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83d3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.706984:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.707022:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660239360:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.707090:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.709735:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660239360:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.709767:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845028.709786:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acae5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83d3 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.709871:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.709874:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.709880:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acae5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83d3 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.709943:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756600, rc 0) 00000100:00100000:1.0:1788845028.710003:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641518464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3148us (3352us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.710011:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 894 00000100:00100000:0.0:1788845028.713452:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.713536:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641518592 00000100:00100000:3.0:1788845028.713547:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641518592 00000100:00100000:3.0:1788845028.713613:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 895 00000100:00100000:3.0:1788845028.713625:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641518592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.713646:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.713678:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287acaec lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83da expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.713714:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.713752:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660239488:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845028.713837:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.716356:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660239488:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845028.716382:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:3.0:1788845028.716405:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287acaec lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83da expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.716426:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845028.716429:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845028.716434:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961c00/0xb2187301287acaec lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83da expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.716457:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961c00, rc 0) 00000100:00100000:3.0:1788845028.716514:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641518592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2892us (3082us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845028.716522:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 895 00000100:00100000:2.0:1788845028.720837:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.721054:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641518720 00000100:00100000:1.0:1788845028.721065:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641518720 00000100:00100000:1.0:1788845028.721126:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 896 00000100:00100000:1.0:1788845028.721135:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641518720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.721157:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.721179:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acaf3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83e1 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.721218:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.721258:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660239616:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.721343:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.725717:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660239616:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.725750:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845028.725772:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acaf3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83e1 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.725793:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.725796:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.725812:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acaf3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83e1 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.725828:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756600, rc 0) 00000100:00100000:1.0:1788845028.725936:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641518720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4801us (5111us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.725945:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 896 00000100:00100000:2.0:1788845028.730730:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.730824:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641518848 00000100:00100000:0.0:1788845028.730835:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641518848 00000100:00100000:0.0:1788845028.730939:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 897 00000100:00100000:0.0:1788845028.730953:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641518848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845028.730978:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845028.731000:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287acafa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83e8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.731039:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.731093:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660239744:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845028.731170:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fe40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.733604:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660239744:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845028.733633:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:0.0:1788845028.733655:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287acafa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83e8 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.733679:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.733683:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845028.733687:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287acafa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83e8 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.733727:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:0.0:1788845028.733784:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641518848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2835us (3064us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845028.733800:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 897 00000100:00100000:1.0:1788845028.737684:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.738119:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641519104 00000100:00100000:0.0:1788845028.738130:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641519104 00000100:00100000:0.0:1788845028.738200:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 898 00000100:00100000:0.0:1788845028.738212:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641519104:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845028.738283:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845028.738311:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0200/0xb2187301287ac81b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8110 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.738330:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.738335:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845028.738692:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.738708:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.738717:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.738750:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.738768:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 14 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.738809:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660239872:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845028.738937:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fe40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.744581:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660239872:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845028.744616:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.745355:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.745456:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641519232 00000100:00100000:2.0:1788845028.745464:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641519232 00010000:00010000:2.0:1788845028.745506:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 15 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.745542:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 330 00000100:00100000:2.0:1788845028.745552:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641519232:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845028.745569:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845028.745575:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 15 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.745592:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 15 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.745632:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f024f2200/0xb2187301287ac876 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8164 expref: 15 pid: 5840 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.745648:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.745653:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.745664:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.745672:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.745698:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845028.745754:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641519232:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 203us (401us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.745762:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 330 00010000:00010000:1.0:1788845028.746477:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.746488:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845028.746626:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967647 00000004:00010000:1.0:1788845028.746683:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:1.0:1788845028.746687:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845028 -> 1788845028 00000004:00010000:1.0:1788845028.746689:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845028 -> 1788845026 00000004:00010000:1.0:1788845028.746691:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845028 -> 1788845028 00000004:00010000:1.0:1788845028.746694:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000004:00010000:1.0:1788845028.746701:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f091ecdf reply state 00000000d4892136 transno 4294967648 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.746711:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845028.746814:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641519104:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8604us (9139us total) trans 4294967648 rc 0/0 00000100:00100000:1.0:1788845028.746822:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 898 00000020:00080000:1.0:1788845028.746827:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845028.752586:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.752605:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.752612:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.752632:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191f000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845028.752637:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.752645:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.752661:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845028.752664:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845028.752669:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.752685:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acb01 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.752685:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.752927:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641519360 00000100:00100000:1.0:1788845028.752937:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641519360 00000100:00100000:1.0:1788845028.752991:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 899 00000100:00100000:1.0:1788845028.753001:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641519360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.753025:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.753042:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83ef expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.753996:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.754011:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.754050:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.754071:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 13 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.755101:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660240000:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845028.755199:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.759713:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.759940:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660240000:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845028.759975:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.759995:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845028.759998:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845028.760003:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.760013:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756c00, rc 0) 00000100:00100000:3.0:1788845028.760064:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641519360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7065us (7385us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.760073:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 899 00000100:00100000:0.0:1788845028.760809:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641519488 00000100:00100000:0.0:1788845028.760817:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641519488 00002000:00100000:0.0:1788845028.760859:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035447800 x1875739641519488/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.760870:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035447800 with x1875739641519488 00010000:00010000:0.0:1788845028.760877:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.760929:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 14 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845028.760937:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a035447800 00000100:00100000:0.0:1788845028.761006:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 21 00000100:00100000:0.0:1788845028.761014:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035447800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641519488:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845028.769406:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:0.0:1788845028.769415:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845028 -> 1788845028 00000004:00010000:0.0:1788845028.769418:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:0.0:1788845028.769427:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845028 -> 1788845028 00000004:00010000:0.0:1788845028.769429:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:0.0:1788845028.769507:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035447800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641519488:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 8489us (9800us total) trans 4294967649 rc 0/0 00000100:00100000:0.0:1788845028.769517:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 21 00002000:00100000:0.0:1788845028.769531:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035447800 x1875739641519488/t4294967649(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/448 e 0 to 0 dl 1788845039 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.769540:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035447800 with x1875739641519488 00010000:00010000:0.0:1788845028.769549:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 16 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.769567:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 16 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845028.769574:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a035447800 00000100:00100000:1.0:1788845028.779167:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.779328:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641519616 00000100:00100000:0.0:1788845028.779342:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641519616 00010000:00010000:0.0:1788845028.779396:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 16 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.779433:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 331 00000100:00100000:0.0:1788845028.779449:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035447100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641519616:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845028.779462:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845028.779487:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 16 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.779505:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 16 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.779541:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f028c2e00/0xb2187301287aca75 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8363 expref: 16 pid: 5841 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.779556:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.779562:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.779572:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.779580:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.779613:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.779627:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660240128:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845028.779701:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fe40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.782806:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660240128:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845028.782838:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845028.782941:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035447100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641519616:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3492us (3783us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.782952:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 331 00000100:00100000:3.0:1788845028.785617:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.785792:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641519744 00000100:00100000:2.0:1788845028.785813:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641519744 00000100:00100000:2.0:1788845028.785875:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 605 00000100:00100000:2.0:1788845028.785930:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641519744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.785952:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.785973:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83f6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.786009:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 15 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.786043:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.786051:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:2.0:1788845028.786057:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:2.0:1788845028.786062:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.786116:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.786136:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 12 pid: 6682 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.786152:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660240256:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845028.786250:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 15 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.786261:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 15 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.786271:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660240384:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845028.786296:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.789508:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660240256:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845028.789525:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660240384:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845028.789562:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.789589:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0000, rc 0) 00000100:00100000:2.0:1788845028.789654:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641519744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3769us (4046us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.789664:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 605 00000100:00100000:1.0:1788845028.792563:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.792957:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641519872 00000100:00100000:2.0:1788845028.792967:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641519872 00010000:00010000:2.0:1788845028.793004:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 13 pid: 6682 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.793045:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 332 00000100:00100000:2.0:1788845028.793054:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035444700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641519872:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845028.793068:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845028.793097:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 13 pid: 6682 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.793115:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 13 pid: 6682 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.793137:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acade lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83cc expref: 13 pid: 6682 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.793181:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845028.793239:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035444700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:2369:x1875739641519872:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 184us (685us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.793246:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 332 00000100:00100000:3.0:1788845028.795015:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.795179:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641520000 00000100:00100000:1.0:1788845028.795188:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641520000 00010000:00010000:1.0:1788845028.795225:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 15 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.795263:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 333 00000100:00100000:1.0:1788845028.795270:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641520000:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845028.795284:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845028.795315:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 15 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.795333:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 15 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.795358:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b8887e8/0xb2187301287acad7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1777664->1781759) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83c5 expref: 15 pid: 7983 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.795376:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:1.0:1788845028.795381:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 5) 00010000:00010000:1.0:1788845028.795391:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.795402:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.795453:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.795471:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660240512:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845028.795568:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.798097:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660240512:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845028.798125:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845028.798171:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641520000:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2900us (3163us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.798183:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 333 00000100:00100000:2.0:1788845028.801287:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.801408:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641520128 00000100:00100000:3.0:1788845028.801420:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641520128 00002000:00010000:3.0:1788845028.801481:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1713082->18446744073709551615) 00002000:00100000:3.0:1788845028.801486:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965880 with x1875739641520128 ext(1713082->18446744073709551615) 00002000:00010000:3.0:1788845028.801499:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965880. 00000100:00100000:3.0:1788845028.801534:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 359 00000100:00100000:3.0:1788845028.801544:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641520128:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845028.801856:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1894431 -> 1713082 00002000:00010000:3.0:1788845028.801860:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845028 -> 1788845028 00002000:00010000:3.0:1788845028.801863:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845028 -> 1788845028 00002000:00010000:3.0:1788845028.801865:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845028 -> 1788845028 00002000:00010000:3.0:1788845028.801868:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:3.0:1788845028.802342:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641520128:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 797us (1060us total) trans 4294967410 rc 0/0 00000100:00100000:3.0:1788845028.802350:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 359 00002000:00010000:3.0:1788845028.802354:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1713082->18446744073709551615) 00002000:00100000:3.0:1788845028.802357:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965880 with x1875739641520128 ext(1713082->18446744073709551615) 00002000:00010000:3.0:1788845028.802362:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965880. 00000100:00100000:3.0:1788845028.809227:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.809420:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641520256 00000100:00100000:2.0:1788845028.809430:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641520256 00000100:00100000:2.0:1788845028.809505:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 900 00000100:00100000:2.0:1788845028.809515:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641520256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.809544:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.809568:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acb16 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a83fd expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.809665:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287acb1d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.809682:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287acb1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.809697:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287acb1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845028.809741:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1713082 00000004:00200000:2.0:1788845028.809746:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845028.809766:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287acb1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.809805:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acb16 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83fd expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.809826:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287acb1d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83fd expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.809835:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.809838:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845028.809842:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0400, rc 0) 00000100:00100000:2.0:1788845028.809956:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641520256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 442us (743us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.809963:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 900 00000100:00100000:1.0:1788845028.813633:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.813722:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641520384 00000100:00100000:0.0:1788845028.813731:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641520384 00000100:00100000:0.0:1788845028.813784:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 901 00000100:00100000:0.0:1788845028.813793:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641520384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845028.813820:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845028.813839:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287acb24 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8404 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.813940:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.813980:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660240640:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845028.814054:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.816880:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.817032:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641520512 00000100:00100000:1.0:1788845028.817043:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641520512 00000100:00100000:1.0:1788845028.817121:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 606 00000100:00100000:1.0:1788845028.817148:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641520512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.817169:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.817188:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d8e6f883/0xb2187301287acb2b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a840b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.817253:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660240768:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.817342:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000004:00010000:0.0:1788845028.818126:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845028 00000100:00100000:0.0:1788845028.818151:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660240640:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845028.818172:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:0.0:1788845028.818189:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287acb24 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8404 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.818210:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.818213:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845028.818217:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287acb24 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8404 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.818232:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:0.0:1788845028.818283:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641520384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4491us (4658us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845028.818292:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 901 00000100:00100000:1.0:1788845028.821451:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660240768:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.821496:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d8e6f883/0xb2187301287acb2b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a840b expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.821522:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8e6f883/0xb2187301287acb2b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a840b expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.821535:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756400, rc 0) 00000100:00100000:1.0:1788845028.821598:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641520512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4453us (4731us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.821608:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 606 00000100:00100000:2.0:1788845028.826220:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.826357:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.826449:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641520768 00000100:00100000:3.0:1788845028.826461:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641520768 00000100:00100000:3.0:1788845028.826554:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 607 00000100:00100000:3.0:1788845028.826565:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641520768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.826585:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.826606:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287acb32 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8419 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.826670:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660240896:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845028.826763:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.827941:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641520640 00000100:00100000:1.0:1788845028.827951:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641520640 00000100:00100000:1.0:1788845028.827997:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 902 00000100:00100000:1.0:1788845028.828005:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641520640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.828022:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.828040:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756200/0xb2187301287acb39 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8412 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.828103:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.828134:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ee300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660241024:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.828200:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.830495:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ee300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660241024:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845028.830526:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:0.0:1788845028.830544:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756200/0xb2187301287acb39 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8412 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.830584:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.830588:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845028.830593:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756200/0xb2187301287acb39 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8412 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.830608:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756200, rc 0) 00000100:00100000:0.0:1788845028.830665:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641520640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2660us (4455us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845028.830673:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 902 00000100:00100000:1.0:1788845028.830810:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660240896:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.830832:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287acb32 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8419 expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.830850:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008df4e277/0xb2187301287acb32 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8419 expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.830860:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961c00, rc 0) 00000100:00100000:1.0:1788845028.830908:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641520768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4345us (4551us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.830925:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 607 00000100:00100000:3.0:1788845028.834750:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.834786:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641520896 00000100:00100000:3.0:1788845028.834809:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641520896 00000100:00100000:3.0:1788845028.834861:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 903 00000100:00100000:3.0:1788845028.834872:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641520896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.834932:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.834955:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb40 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8420 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.835048:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.835064:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.835076:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845028.835124:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1713082 00000004:00200000:3.0:1788845028.835128:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845028.835147:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.835174:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb40 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.835192:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.835201:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845028.835204:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845028.835207:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961600, rc 0) 00000100:00100000:3.0:1788845028.835609:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641520896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 736us (867us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.835617:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 903 00000100:00100000:0.0:1788845028.839740:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.839805:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641521024 00000100:00100000:1.0:1788845028.839816:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641521024 00000100:00100000:0.0:1788845028.839864:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.839898:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 904 00000100:00100000:1.0:1788845028.839909:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641521024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.839942:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.839966:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287acb4e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8427 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.840018:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.840051:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660241152:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.840119:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.840184:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641521152 00000100:00100000:2.0:1788845028.840188:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641521152 00000100:00100000:2.0:1788845028.840224:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 608 00000100:00100000:2.0:1788845028.840232:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641521152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.840241:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.840254:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287acb55 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a842e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.840302:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660241280:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.840345:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.844430:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660241280:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.844545:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287acb55 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a842e expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.844575:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287acb55 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a842e expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.844590:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1800, rc 0) 00000100:00100000:1.0:1788845028.844636:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660241152:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.844655:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:1.0:1788845028.844668:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287acb4e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8427 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.844682:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.844685:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00000100:00100000:2.0:1788845028.844687:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641521152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4455us (4893us total) trans 0 rc 301/301 00010000:00010000:1.0:1788845028.844690:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287acb4e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8427 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.844696:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 608 00010000:00010000:1.0:1788845028.844701:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756400, rc 0) 00000100:00100000:1.0:1788845028.844724:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641521024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4818us (4997us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.844731:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 904 00000100:00100000:0.0:1788845028.848657:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.848789:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.849055:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641521408 00000100:00100000:1.0:1788845028.849065:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641521408 00000100:00100000:1.0:1788845028.849128:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 609 00000100:00100000:1.0:1788845028.849138:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641521408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.849157:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.849174:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acb5c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a843c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.849251:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660241408:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.849459:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.851945:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660241408:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.851991:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acb5c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a843c expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.852018:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acb5c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a843c expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.852032:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757c00, rc 0) 00000100:00100000:1.0:1788845028.852098:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641521408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2960us (3307us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.852106:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 609 00000100:00100000:2.0:1788845028.853611:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641521280 00000100:00100000:2.0:1788845028.853626:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641521280 00000100:00100000:2.0:1788845028.853731:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 905 00000100:00100000:2.0:1788845028.853742:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641521280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.853763:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.853784:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acb63 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8435 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.853833:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.853929:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660241536:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.854006:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.859130:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660241536:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845028.859158:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1006713 00010000:00010000:2.0:1788845028.859179:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acb63 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8435 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.859200:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.859203:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845028.859208:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acb63 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8435 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.859222:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:2.0:1788845028.859300:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641521280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5560us (10658us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.859309:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 905 00000100:00100000:3.0:1788845028.862586:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.862678:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641521664 00000100:00100000:2.0:1788845028.862688:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641521664 00000100:00100000:2.0:1788845028.862750:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 906 00000100:00100000:2.0:1788845028.862760:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641521664:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845028.862823:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845028.862850:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0400/0xb2187301287acb1d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83fd expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.862870:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845028.862929:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845028.863133:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.863149:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.863157:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.863192:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.863210:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 12 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.863226:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660241664:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845028.863305:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.867467:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.867574:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660241664:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845028.867607:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.867728:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641521792 00000100:00100000:1.0:1788845028.867736:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641521792 00010000:00010000:1.0:1788845028.867771:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 13 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.867818:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 334 00000100:00100000:1.0:1788845028.867829:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008914450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641521792:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845028.867843:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845028.867850:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 13 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.867878:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 13 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.867946:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961600/0xb2187301287acb47 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8420 expref: 13 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.867963:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.867968:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.867979:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.867987:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.868007:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845028.868051:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008914450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641521792:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 225us (602us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.868059:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 334 00010000:00010000:2.0:1788845028.868207:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.868215:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845028.868354:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967650 00000004:00010000:2.0:1788845028.868404:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 1006713 00000004:00010000:2.0:1788845028.868413:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845028 -> 1788845028 00000004:00010000:2.0:1788845028.868416:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845028 -> 1788845026 00000004:00010000:2.0:1788845028.868419:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845028 -> 1788845028 00000004:00010000:2.0:1788845028.868421:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000004:00010000:2.0:1788845028.868429:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000033c2e096 reply state 00000000ec1f8fe2 transno 4294967651 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.868448:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845028.868525:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641521664:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5766us (5950us total) trans 4294967651 rc 0/0 00000100:00100000:2.0:1788845028.868533:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 906 00000020:00080000:2.0:1788845028.868536:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845028.872934:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.872952:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.872960:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.872986:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845028.872992:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.873001:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.873017:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.873019:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845028.873024:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.873030:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287acb6a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.873185:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.873466:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641521920 00000100:00100000:3.0:1788845028.873476:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641521920 00000100:00100000:3.0:1788845028.873529:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 907 00000100:00100000:3.0:1788845028.873538:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641521920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.873563:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.873584:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8443 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.873610:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.873627:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.873662:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.873684:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 11 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.873701:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660241792:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845028.873804:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.876624:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660241792:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845028.876661:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.876683:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.876686:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845028.876691:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.876702:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01960a00, rc 0) 00000100:00100000:0.0:1788845028.876779:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641521920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3242us (3598us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845028.876787:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 907 00000100:00100000:1.0:1788845028.880098:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.880179:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641522048 00000100:00100000:3.0:1788845028.880190:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641522048 00010000:00010000:3.0:1788845028.880231:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.880285:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 335 00000100:00100000:3.0:1788845028.880295:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641522048:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845028.880311:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845028.880342:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.880362:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.880386:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756c00/0xb2187301287acb08 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83ef expref: 12 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.880419:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845028.880427:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.880449:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.880460:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.880498:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845028.880514:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660241920:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845028.880597:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.885405:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.885488:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641522176 00000100:00100000:1.0:1788845028.885499:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641522176 00000100:00100000:1.0:1788845028.885563:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 610 00000100:00100000:1.0:1788845028.885574:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:14891:x1875739641522176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.885596:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.885616:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a844a expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.885647:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.885657:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(5 5) 00010000:00010000:1.0:1788845028.885678:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 5) 00010000:00010000:1.0:1788845028.885684:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.885724:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.885744:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 14 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.885758:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660242048:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845028.885844:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.888171:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660242048:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845028.888208:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.888233:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757000, rc 0) 00000100:00100000:2.0:1788845028.888304:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641522176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2730us (2911us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.888312:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 610 00000100:00100000:0.0:1788845028.889209:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660241920:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845028.889226:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845028.889267:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641522048:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8973us (9178us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845028.889277:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 335 00000100:00100000:1.0:1788845028.892194:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.892300:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641522304 00000100:00100000:2.0:1788845028.892309:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641522304 00010000:00010000:2.0:1788845028.892345:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 15 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.892382:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 336 00000100:00100000:2.0:1788845028.892392:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641522304:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845028.892423:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845028.892647:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 15 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.892662:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 15 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.892684:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000943564dd/0xb2187301287acb0f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a83f6 expref: 15 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.892697:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 5) 00010000:00010000:2.0:1788845028.892701:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 5) 00010000:00010000:2.0:1788845028.892709:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.892716:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.892747:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.892759:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660242176:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845028.892831:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845028.898816:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.898862:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.898979:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641522432 00000100:00100000:0.0:1788845028.898988:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641522432 00002000:00010000:0.0:1788845028.899026:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845028.899031:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c5180 with x1875739641522432 00000100:00100000:0.0:1788845028.899071:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 562 00000100:00100000:0.0:1788845028.899080:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641522432:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845028.899322:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1006713 -> 983102 00000004:00010000:0.0:1788845028.899325:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845028 -> 1788845028 00000004:00010000:0.0:1788845028.899327:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:0.0:1788845028.899329:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845028 -> 1788845028 00000004:00010000:0.0:1788845028.899330:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:0.0:1788845028.899388:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641522432:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 308us (525us total) trans 4294967652 rc 0/0 00000100:00100000:0.0:1788845028.899395:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 562 00002000:00010000:0.0:1788845028.899398:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845028.899400:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c5180 with x1875739641522432 00000100:00100000:1.0:1788845028.899646:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641522560 00000100:00100000:1.0:1788845028.899651:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641522560 00002000:00010000:1.0:1788845028.899669:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845028.899673:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c6300 with x1875739641522560 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845028.899680:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c6300. 00000100:00100000:1.0:1788845028.899701:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 360 00000100:00100000:1.0:1788845028.899706:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641522560:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845028.900294:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1713082 -> 0 00002000:00010000:1.0:1788845028.900299:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845028 -> 1788845028 00002000:00010000:1.0:1788845028.900301:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845028 -> 1788845028 00002000:00010000:1.0:1788845028.900303:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845028 -> 1788845028 00002000:00010000:1.0:1788845028.900305:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:1.0:1788845028.900337:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641522560:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 629us (1531us total) trans 4294967411 rc 0/0 00000100:00100000:1.0:1788845028.900355:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 360 00002000:00010000:1.0:1788845028.900358:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845028.900361:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c6300 with x1875739641522560 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845028.900366:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c6300. 00000100:00100000:2.0:1788845028.901609:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660242176:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845028.901648:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845028.901691:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641522304:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9300us (9507us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845028.901701:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 336 00000100:00100000:3.0:1788845028.904615:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.905290:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641522688 00000100:00100000:1.0:1788845028.905300:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641522688 00000100:00100000:1.0:1788845028.905357:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 908 00000100:00100000:1.0:1788845028.905365:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641522688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.905386:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.905409:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757a00/0xb2187301287acb7f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8451 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.905599:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287acb86 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.905612:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287acb86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.905621:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287acb86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845028.905642:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 983102 00000004:00200000:1.0:1788845028.905645:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845028.905660:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287acb86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.905680:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757a00/0xb2187301287acb7f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8451 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.905714:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287acb86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8451 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.905720:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.905722:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845028.905724:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756a00, rc 0) 00000100:00100000:1.0:1788845028.905820:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641522688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 454us (1211us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845028.905827:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 908 00000100:00100000:3.0:1788845028.912238:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.913826:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.915120:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641522816 00000100:00100000:0.0:1788845028.915145:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641522816 00000100:00100000:0.0:1788845028.915239:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 909 00000100:00100000:0.0:1788845028.915250:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641522816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845028.915275:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845028.915297:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287acb8d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8458 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.915336:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.915377:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660242304:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845028.915639:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.916227:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641522944 00000100:00100000:1.0:1788845028.916234:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641522944 00000100:00100000:1.0:1788845028.916799:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 611 00000100:00100000:1.0:1788845028.916812:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641522944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.916827:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.916842:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287acb94 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a845f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.916950:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660242432:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.917022:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000004:00010000:1.0:1788845028.917780:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845028 00000100:00100000:1.0:1788845028.917818:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660242304:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.917837:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:1.0:1788845028.917853:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287acb8d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8458 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.917942:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.917946:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.917952:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287acb8d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8458 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.917966:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ec00, rc 0) 00000100:00100000:1.0:1788845028.918008:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641522816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2759us (5781us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.918016:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 909 00000100:00100000:1.0:1788845028.920722:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660242432:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.920756:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287acb94 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a845f expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.920774:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287acb94 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a845f expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.920783:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756e00, rc 0) 00000100:00100000:1.0:1788845028.920857:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641522944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4049us (7040us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.920864:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 611 00000100:00100000:0.0:1788845028.923300:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.923687:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641523072 00000100:00100000:2.0:1788845028.923758:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641523072 00000100:00100000:3.0:1788845028.923805:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.923812:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 910 00000100:00100000:2.0:1788845028.923822:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641523072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.923843:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.923863:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acb9b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8466 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.923947:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.923992:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660242560:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845028.924284:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641523200 00000100:00100000:0.0:1788845028.924288:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641523200 00000100:00100000:0.0:1788845028.924387:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 612 00000100:00100000:0.0:1788845028.924395:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641523200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845028.924405:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845028.924416:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287acba2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a846d expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845028.924527:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660242688:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.924629:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845028.924670:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845028.926317:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660242688:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.926423:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287acba2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a846d expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.926522:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287acba2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a846d expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.926537:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:0.0:1788845028.926653:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660242560:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845028.926666:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:0.0:1788845028.926679:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acb9b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8466 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.926721:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845028.926724:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845028.926729:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acb9b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8466 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845028.926740:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:2.0:1788845028.927012:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641523200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2616us (3206us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.927021:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 612 00000100:00100000:0.0:1788845028.927567:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641523072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3744us (4273us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845028.927575:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 910 00000100:00100000:0.0:1788845028.929422:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.929518:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641523328 00000100:00100000:3.0:1788845028.929528:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641523328 00000100:00100000:3.0:1788845028.929591:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 911 00000100:00100000:3.0:1788845028.929601:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641523328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845028.929646:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845028.929669:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287acba9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8474 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.929749:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.929762:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.929772:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845028.929849:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 983102 00000004:00200000:3.0:1788845028.929853:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845028.929871:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.929924:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961a00/0xb2187301287acba9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.929941:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845028.929949:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845028.929952:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845028.929955:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01961800, rc 0) 00000100:00100000:3.0:1788845028.930013:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641523328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 414us (599us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845028.930020:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 911 00000100:00100000:3.0:1788845028.934582:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845028.934627:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.934747:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641523584 00000100:00100000:2.0:1788845028.934757:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641523584 00000100:00100000:2.0:1788845028.934818:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 613 00000100:00100000:2.0:1788845028.934829:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641523584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.934848:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.934865:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287acbb7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8482 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.934954:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660242816:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.935032:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.936003:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641523456 00000100:00100000:1.0:1788845028.936011:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641523456 00000100:00100000:1.0:1788845028.936062:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 912 00000100:00100000:1.0:1788845028.936071:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641523456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.936100:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.936116:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757a00/0xb2187301287acbbe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a847b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.936374:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.936408:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660242944:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.936485:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00002000:00010000:1.0:1788845028.937263:0:6684:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1951982 00000100:00100000:1.0:1788845028.937291:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660242816:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845028.937323:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287acbb7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8482 expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.937346:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287acbb7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8482 expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.937359:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:1.0:1788845028.937461:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641523584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2634us (2833us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.937470:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 613 00000100:00100000:1.0:1788845028.939762:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660242944:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.939789:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:1.0:1788845028.939809:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757a00/0xb2187301287acbbe lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a847b expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.939850:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.939853:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.939858:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757a00/0xb2187301287acbbe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a847b expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.939872:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757a00, rc 0) 00000100:00100000:1.0:1788845028.940295:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641523456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3895us (5392us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.940306:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 912 00000100:00100000:0.0:1788845028.942182:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.942222:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845028.942314:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641523712 00000100:00100000:1.0:1788845028.942324:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641523712 00000100:00100000:1.0:1788845028.942379:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 913 00000100:00100000:1.0:1788845028.942389:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641523712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845028.942411:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845028.942431:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acbc5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8489 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.942478:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845028.942520:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660243072:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845028.942598:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845028.944118:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660243072:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845028.944153:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:1.0:1788845028.944171:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acbc5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8489 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.944208:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845028.944211:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845028.944215:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756600/0xb2187301287acbc5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8489 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845028.944227:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756600, rc 0) 00000100:00100000:1.0:1788845028.944278:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641523712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1890us (2104us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845028.944287:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 913 00000100:00100000:2.0:1788845028.944514:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641523840 00000100:00100000:2.0:1788845028.944521:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641523840 00000100:00100000:2.0:1788845028.944607:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 614 00000100:00100000:2.0:1788845028.944618:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641523840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845028.944634:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845028.944650:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287acbcc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8490 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845028.944713:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660243200:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845028.944776:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788845028.946211:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845028.949637:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660243200:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845028.949680:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287acbcc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8490 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.949733:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c3c34d0/0xb2187301287acbcc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8490 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845028.949747:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0600, rc 0) 00000100:00100000:2.0:1788845028.949810:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641523840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5193us (7586us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845028.949820:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 614 00000100:00100000:3.0:1788845028.970462:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845028.970955:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641523968 00000100:00100000:2.0:1788845028.970978:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641523968 00002000:00100000:2.0:1788845028.971031:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03412300 x1875739641523968/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845028.971045:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03412300 with x1875739641523968 00000100:00100000:2.0:1788845028.971088:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 563 00000100:00100000:2.0:1788845028.971098:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2367:x1875739641523968:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845028.975855:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845028.978579:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641524096 00000100:00100000:0.0:1788845028.978595:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641524096 00002000:00100000:0.0:1788845028.978649:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1892351) req@ffff949f03410000 x1875739641524096/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/0 e 0 to 0 dl 1788845039 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845028.978661:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03410000 with x1875739641524096 ext(1048576->1892351) 00002000:00010000:0.0:1788845028.978667:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03410000 00000100:00100000:0.0:1788845028.978694:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 361 00000100:00100000:0.0:1788845028.978705:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641524096:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845029.000725:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2367:x1875739641523968:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 29616us (30262us total) trans 0 rc 987136/987136 00000100:00100000:2.0:1788845029.000741:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 563 00002000:00100000:2.0:1788845029.000754:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03412300 x1875739641523968/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/440 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 987136/987136 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.000764:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03412300 with x1875739641523968 00000100:00100000:1.0:1788845029.002642:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641524096:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 23929us (26785us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845029.002655:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 361 00002000:00100000:1.0:1788845029.002667:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1892351) req@ffff949f03410000 x1875739641524096/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:134/0 lens 488/440 e 0 to 0 dl 1788845039 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845029.002678:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03410000 with x1875739641524096 ext(1048576->1892351) 00002000:00010000:1.0:1788845029.002684:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03410000 00000100:00100000:3.0:1788845029.010426:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.010487:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.010642:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641524224 00000100:00100000:0.0:1788845029.010652:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641524224 00000100:00100000:0.0:1788845029.010727:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 914 00000100:00100000:0.0:1788845029.010738:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641524224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.010784:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.010813:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287acbd3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8497 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.010853:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.010933:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660243328:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.011024:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.011556:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641524352 00000100:00100000:1.0:1788845029.011562:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641524352 00000100:00100000:1.0:1788845029.011610:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 615 00000100:00100000:1.0:1788845029.011619:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641524352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.011630:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.011643:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287acbda lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a849e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.011694:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660243456:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.011743:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00080000:2.0:1788845029.011756:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788845029.011763:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:1.0:1788845029.015746:0:7983:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845028 -> 1788845029 00002000:00010000:1.0:1788845029.015757:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845028 -> 1788845029 00000100:00100000:1.0:1788845029.015783:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660243456:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.015837:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287acbda lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a849e expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.015863:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287acbda lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a849e expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.015876:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756600, rc 0) 00000100:00100000:1.0:1788845029.015998:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641524352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4380us (5509us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.016007:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 615 00000004:00010000:0.0:1788845029.019619:0:11036:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845028 -> 1788845029 00000004:00010000:0.0:1788845029.019630:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845028 -> 1788845029 00000100:00100000:0.0:1788845029.019656:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660243328:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845029.019681:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:0.0:1788845029.019703:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287acbd3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8497 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.019725:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.019728:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.019733:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287acbd3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8497 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.019749:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e000, rc 0) 00000100:00100000:0.0:1788845029.019861:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641524224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9125us (9443us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.019869:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 914 00000100:00100000:0.0:1788845029.028248:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.028407:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.028489:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641524608 00000100:00100000:2.0:1788845029.028506:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641524608 00000100:00100000:3.0:1788845029.028608:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641524480 00000100:00100000:3.0:1788845029.028614:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641524480 00000100:00100000:2.0:1788845029.028651:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 616 00000100:00100000:2.0:1788845029.028664:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641524608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.028686:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1788845029.028693:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 915 00000100:00100000:3.0:1788845029.028704:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641524480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.028710:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287acbe1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84ac expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.028719:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.028735:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287acbe8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84a5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.028771:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.028777:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660243584:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.028806:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03412680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660243712:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.029131:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.031180:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.032256:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660243584:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845029.032296:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287acbe1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84ac expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.032325:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287acbe1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84ac expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.032340:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:3.0:1788845029.032590:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03412680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660243712:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845029.032604:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:3.0:1788845029.032616:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287acbe8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84a5 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.032628:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845029.032631:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845029.032635:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287acbe8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84a5 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.032663:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:0.0:1788845029.032706:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641524608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4044us (4297us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.032715:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 616 00000100:00100000:3.0:1788845029.036186:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641524480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7470us (7935us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.036200:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 915 00000100:00100000:0.0:1788845029.053181:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.053225:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.053389:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641524736 00000100:00100000:3.0:1788845029.053399:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641524736 00000100:00100000:3.0:1788845029.053461:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 916 00000100:00100000:3.0:1788845029.053472:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641524736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.053493:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.053512:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287acbef lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84b3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.053549:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.053596:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03412680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660243840:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.053680:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.054258:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641524864 00000100:00100000:1.0:1788845029.054264:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641524864 00000100:00100000:1.0:1788845029.054303:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 617 00000100:00100000:1.0:1788845029.054312:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641524864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.054323:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.054336:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acbf6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84ba expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.054387:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660243968:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.054454:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.056354:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03412680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660243840:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845029.056450:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:3.0:1788845029.056469:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287acbef lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84b3 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.056489:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845029.056492:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845029.056498:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287acbef lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84b3 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.056526:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:3.0:1788845029.056628:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641524736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3158us (3458us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.064329:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 916 00000100:00100000:1.0:1788845029.071327:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660243968:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.071376:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acbf6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84ba expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.071403:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acbf6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84ba expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.071417:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757a00, rc 0) 00000100:00100000:1.0:1788845029.071485:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641524864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17173us (18258us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.071493:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 617 00000100:00100000:0.0:1788845029.081440:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.081491:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.081616:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641524992 00000100:00100000:1.0:1788845029.081628:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641524992 00000100:00100000:1.0:1788845029.081689:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 917 00000100:00100000:1.0:1788845029.081699:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641524992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.081720:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.081740:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acbfd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84c1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.081775:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.081814:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660244096:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.081925:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.082330:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641525120 00000100:00100000:2.0:1788845029.082336:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641525120 00000100:00100000:2.0:1788845029.082383:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 618 00000100:00100000:2.0:1788845029.082391:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641525120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.082402:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.082415:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287acc04 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84c8 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.082454:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660244224:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.082502:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.088017:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660244096:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845029.088044:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:1.0:1788845029.088062:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acbfd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84c1 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.088083:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.088087:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845029.088091:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acbfd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84c1 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.088108:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757600, rc 0) 00000100:00100000:1.0:1788845029.088159:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641524992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6461us (6729us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.088169:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 917 00000100:00100000:2.0:1788845029.089025:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660244224:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845029.089052:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287acc04 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84c8 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.089072:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287acc04 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84c8 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.089105:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:2.0:1788845029.089135:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641525120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6745us (7643us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.089142:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 618 00000100:00100000:3.0:1788845029.093386:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.093441:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.093518:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641525376 00000100:00100000:1.0:1788845029.093529:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641525376 00000100:00100000:1.0:1788845029.093607:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 619 00000100:00100000:1.0:1788845029.093622:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641525376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.093645:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.093668:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc0b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84d6 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.093760:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660244352:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.093861:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.094557:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641525248 00000100:00100000:0.0:1788845029.094565:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641525248 00000100:00100000:0.0:1788845029.094616:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 918 00000100:00100000:0.0:1788845029.094626:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641525248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.094640:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.094656:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acc12 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84cf expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.094688:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.094750:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660244480:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.094816:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.097986:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660244480:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845029.098015:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:0.0:1788845029.098034:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acc12 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84cf expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.098056:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.098059:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.098064:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acc12 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84cf expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.098079:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e800, rc 0) 00000100:00100000:0.0:1788845029.098143:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641525248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3518us (4769us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.098152:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 918 00000100:00100000:1.0:1788845029.100012:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660244352:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.100049:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc0b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84d6 expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.100086:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc0b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84d6 expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.100099:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757a00, rc 0) 00000100:00100000:1.0:1788845029.100152:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641525376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6535us (6711us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.100160:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 619 00000100:00100000:2.0:1788845029.106664:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.106711:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.106730:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641525504 00000100:00100000:0.0:1788845029.106741:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641525504 00000100:00100000:1.0:1788845029.106836:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641525632 00000100:00100000:1.0:1788845029.106842:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641525632 00000100:00100000:1.0:1788845029.106934:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 620 00000100:00100000:1.0:1788845029.106947:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641525632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.106969:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.106990:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005650e704/0xb2187301287acc19 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84e4 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.107055:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660244608:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.107157:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.107295:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 919 00000100:00100000:0.0:1788845029.107307:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641525504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.107321:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.107340:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287acc20 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84dd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.107374:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.107416:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660244736:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.107467:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.110693:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660244608:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.110747:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005650e704/0xb2187301287acc19 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84e4 expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.110773:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005650e704/0xb2187301287acc19 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84e4 expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.110807:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757600, rc 0) 00000100:00100000:1.0:1788845029.110875:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641525632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3931us (4162us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.110927:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 620 00000100:00100000:0.0:1788845029.111314:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660244736:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845029.111332:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 983102 00010000:00010000:0.0:1788845029.111346:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287acc20 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84dd expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.111362:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.111366:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.111370:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287acc20 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84dd expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.111382:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f400, rc 0) 00000100:00100000:0.0:1788845029.111538:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641525504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4232us (4881us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.111546:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 919 00000100:00100000:2.0:1788845029.115868:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.116038:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641525888 00000100:00100000:1.0:1788845029.116048:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641525888 00000100:00100000:1.0:1788845029.116105:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 920 00000100:00100000:1.0:1788845029.116115:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641525888:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845029.116167:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845029.116194:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287acb86 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8451 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.116212:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.116217:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845029.116618:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.116635:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.116643:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.116677:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.116726:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 19 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.116741:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660244864:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845029.116832:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c929c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.119369:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.119471:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641526016 00000100:00100000:2.0:1788845029.119481:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641526016 00010000:00010000:2.0:1788845029.119540:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 20 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.119582:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 337 00000100:00100000:2.0:1788845029.119594:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641526016:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845029.119609:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845029.119617:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 20 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.119619:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660244864:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845029.119645:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 20 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1788845029.119687:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788845029.122377:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.128637:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01961800/0xb2187301287acbb0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8474 expref: 20 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.128667:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.128691:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.128703:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.128717:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.128751:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845029.128819:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641526016:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9226us (9458us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.128830:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 337 00010000:00010000:1.0:1788845029.129286:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.129296:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845029.129417:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967653 00000004:00010000:1.0:1788845029.129467:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 983102 -> 983102 00000004:00010000:1.0:1788845029.129470:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:1.0:1788845029.129473:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845028 -> 1788845026 00000004:00010000:1.0:1788845029.129475:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:1.0:1788845029.129477:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000004:00010000:1.0:1788845029.129483:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007d2a6ede reply state 0000000001b8f83b transno 4294967654 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.129491:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845029.129589:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641525888:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13474us (13729us total) trans 4294967654 rc 0/0 00000100:00100000:1.0:1788845029.129609:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 920 00000020:00080000:1.0:1788845029.129613:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845029.132133:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845029.132136:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.132149:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.132157:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.132164:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02757600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845029.132182:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.132191:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.132209:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.132212:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845029.132217:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.132223:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757600/0xb2187301287acc27 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.132357:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641526144 00000100:00100000:2.0:1788845029.132367:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641526144 00000100:00100000:2.0:1788845029.132412:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 921 00000100:00100000:2.0:1788845029.132422:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641526144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.132445:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.132464:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84eb expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.132491:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.132500:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.132531:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.132559:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 18 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.132730:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660244992:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845029.132814:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.142518:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660244992:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845029.142558:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.142698:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.142701:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.142706:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.142717:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845029.143169:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641526144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10747us (11043us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.143178:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 921 00000100:00100000:0.0:1788845029.145931:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.146179:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641526272 00000100:00100000:3.0:1788845029.146189:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641526272 00002000:00100000:3.0:1788845029.146244:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48380 x1875739641526272/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 8680/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.146255:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48380 with x1875739641526272 00010000:00010000:3.0:1788845029.146265:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.146285:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 19 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845029.146292:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a02da48380 00000100:00100000:3.0:1788845029.146317:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 22 00000100:00100000:3.0:1788845029.146327:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641526272:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845029.147188:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 983102 -> 983102 00000004:00010000:3.0:1788845029.147194:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:3.0:1788845029.147196:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:3.0:1788845029.147203:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:3.0:1788845029.147206:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:3.0:1788845029.147283:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641526272:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 956us (1406us total) trans 4294967655 rc 0/0 00000100:00100000:3.0:1788845029.147291:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 22 00002000:00100000:3.0:1788845029.147299:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48380 x1875739641526272/t4294967655(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 8680/448 e 0 to 0 dl 1788845040 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.147306:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48380 with x1875739641526272 00010000:00010000:3.0:1788845029.147312:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 21 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.147326:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 21 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845029.147332:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a02da48380 00000100:00100000:2.0:1788845029.159071:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.159157:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641526400 00000100:00100000:0.0:1788845029.159167:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641526400 00010000:00010000:0.0:1788845029.159207:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 21 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.159247:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 338 00000100:00100000:0.0:1788845029.159257:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641526400:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845029.159272:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845029.159296:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 21 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.159313:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 21 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.159335:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01960a00/0xb2187301287acb71 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8443 expref: 21 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.159366:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.159373:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.159382:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.159391:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.159427:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.159441:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660245120:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845029.159659:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.172059:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.172226:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660245120:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845029.172252:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845029.172311:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641526400:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 13055us (13249us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.172323:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 338 00000100:00100000:1.0:1788845029.172431:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641526528 00000100:00100000:1.0:1788845029.172439:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641526528 00000100:00100000:1.0:1788845029.172492:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 621 00000100:00100000:1.0:1788845029.172501:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641526528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.172518:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.172538:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84f2 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.172671:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.172682:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 5) 00010000:00010000:1.0:1788845029.172686:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 5) 00010000:00010000:1.0:1788845029.172691:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.172726:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.172744:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 13 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.172758:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660245248:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845029.172815:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.178102:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660245248:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845029.178142:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.178164:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757a00, rc 0) 00000100:00100000:1.0:1788845029.178300:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.178338:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641526528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5836us (6288us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.178346:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 621 00000100:00100000:3.0:1788845029.178424:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641526656 00000100:00100000:3.0:1788845029.178433:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641526656 00002000:00100000:3.0:1788845029.178492:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1892352->1951981) req@ffff94a0360ee300 x1875739641526656/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.178501:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ee300 with x1875739641526656 ext(1892352->1951981) 00010000:00010000:3.0:1788845029.178508:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 14 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.178525:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 14 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845029.178553:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a0360ee300 00000100:00100000:3.0:1788845029.178578:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 17 00000100:00100000:3.0:1788845029.178587:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641526656:12345-192.168.201.29@tcp:4:fsx.0 00000100:00080000:2.0:1788845029.178744:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845029.209938:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641526656:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 31298us (32000us total) trans 4294967412 rc 0/0 00000100:00100000:1.0:1788845029.209954:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 17 00002000:00100000:1.0:1788845029.209968:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1892352->1951981) req@ffff94a0360ee300 x1875739641526656/t4294967412(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/448 e 0 to 0 dl 1788845040 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845029.209981:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ee300 with x1875739641526656 ext(1892352->1951981) 00010000:00010000:1.0:1788845029.209993:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 16 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.210018:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 16 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845029.210027:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a0360ee300 00000100:00100000:2.0:1788845029.215195:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.215301:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641526784 00000100:00100000:3.0:1788845029.215312:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641526784 00010000:00010000:3.0:1788845029.215354:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 16 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.215399:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 339 00000100:00100000:3.0:1788845029.215410:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641526784:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845029.215425:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845029.215461:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 16 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.215481:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 16 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.215503:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287acb78 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a844a expref: 16 pid: 6682 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.215520:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 5) 00010000:00010000:3.0:1788845029.215525:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 5) 00010000:00010000:3.0:1788845029.215532:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.215557:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.215598:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.215613:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660245376:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845029.215698:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.223932:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660245376:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845029.223977:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845029.224034:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641526784:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8626us (8847us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845029.224046:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 339 00000100:00100000:0.0:1788845029.225697:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.225758:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.225770:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641526912 00000100:00100000:2.0:1788845029.225780:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641526912 00002000:00010000:2.0:1788845029.225822:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845029.225827:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4aa00 with x1875739641526912 00000100:00100000:2.0:1788845029.225868:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 564 00000100:00100000:2.0:1788845029.225879:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641526912:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:2.0:1788845029.226287:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 983102 -> 440975 00000004:00010000:2.0:1788845029.226291:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:2.0:1788845029.226293:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845029.226295:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:2.0:1788845029.226297:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:2.0:1788845029.226353:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641526912:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 477us (662us total) trans 4294967656 rc 0/0 00000100:00100000:2.0:1788845029.226360:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 564 00002000:00010000:2.0:1788845029.226363:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845029.226365:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4aa00 with x1875739641526912 00000100:00100000:1.0:1788845029.227557:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641527040 00000100:00100000:1.0:1788845029.227568:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641527040 00002000:00010000:1.0:1788845029.227604:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845029.227608:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da48700 with x1875739641527040 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845029.227617:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da48700. 00000100:00100000:1.0:1788845029.227655:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 362 00000100:00100000:1.0:1788845029.227664:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641527040:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845029.228037:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1951982 -> 0 00002000:00010000:1.0:1788845029.228041:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00002000:00010000:1.0:1788845029.228044:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845028 -> 1788845028 00002000:00010000:1.0:1788845029.228070:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00002000:00010000:1.0:1788845029.228073:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:1.0:1788845029.228128:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641527040:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 465us (2370us total) trans 4294967413 rc 0/0 00000100:00100000:1.0:1788845029.228138:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 362 00002000:00010000:1.0:1788845029.228143:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845029.228146:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da48700 with x1875739641527040 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845029.228152:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da48700. 00000100:00100000:0.0:1788845029.233315:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.233486:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641527168 00000100:00100000:2.0:1788845029.233499:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641527168 00000100:00100000:2.0:1788845029.233570:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 922 00000100:00100000:2.0:1788845029.233583:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641527168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.233610:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.233631:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc3c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a84f9 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.233742:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acc43 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.233760:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acc43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.233772:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acc43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845029.233810:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 440975 00000004:00200000:2.0:1788845029.233815:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845029.233833:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acc43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.233860:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc3c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f9 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.233878:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acc43 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f9 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.233951:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.233955:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845029.233971:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:2.0:1788845029.234039:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641527168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 460us (734us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.234047:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 922 00000100:00100000:3.0:1788845029.240181:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.240285:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641527296 00000100:00100000:0.0:1788845029.240295:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641527296 00000100:00100000:0.0:1788845029.240363:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 923 00000100:00100000:0.0:1788845029.240373:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641527296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.240395:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.240414:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287acc4a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8500 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.240467:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.240505:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660245504:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.240604:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.241105:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000004:00010000:0.0:1788845029.245731:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845028 00000100:00100000:0.0:1788845029.245772:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660245504:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845029.245799:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 440975 00010000:00010000:0.0:1788845029.245820:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287acc4a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8500 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.245841:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.245844:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.245850:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287acc4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8500 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.245866:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ec00, rc 0) 00000100:00100000:0.0:1788845029.246050:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641527296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5677us (5877us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.246060:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 923 00000100:00100000:2.0:1788845029.246289:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641527424 00000100:00100000:2.0:1788845029.246297:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641527424 00000100:00100000:2.0:1788845029.246826:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 622 00000100:00100000:2.0:1788845029.246839:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641527424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.246859:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.246876:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287acc51 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8507 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.246984:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660245632:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.247051:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.255096:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660245632:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845029.255140:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287acc51 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8507 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.255165:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000083ab5f4/0xb2187301287acc51 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8507 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.255179:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845029.255246:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641527424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8410us (14141us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.255255:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 622 00000100:00100000:0.0:1788845029.263448:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.263492:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.263590:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641527552 00000100:00100000:2.0:1788845029.263601:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641527552 00000100:00100000:2.0:1788845029.263663:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 924 00000100:00100000:2.0:1788845029.263673:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641527552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.263693:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.263716:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc58 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a850e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.263754:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.263805:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660245760:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.263919:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.264303:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641527680 00000100:00100000:1.0:1788845029.264308:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641527680 00000100:00100000:1.0:1788845029.264352:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 623 00000100:00100000:1.0:1788845029.264359:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641527680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.264369:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.264380:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009eed212f/0xb2187301287acc5f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8515 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.264414:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660245888:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.264457:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a010e58e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.266235:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660245888:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.266271:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009eed212f/0xb2187301287acc5f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8515 expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.266294:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009eed212f/0xb2187301287acc5f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8515 expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.266325:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756800, rc 0) 00000100:00100000:1.0:1788845029.266375:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641527680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2016us (2882us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.266382:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 623 00000100:00100000:2.0:1788845029.267104:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660245760:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.267121:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 440975 00010000:00010000:2.0:1788845029.267134:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc58 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a850e expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.267149:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.267151:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.267156:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc58 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a850e expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.267167:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845029.267206:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641527552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3535us (3768us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.267213:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 924 00000100:00100000:3.0:1788845029.270285:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.270375:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641527936 00000100:00100000:1.0:1788845029.270385:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641527936 00000100:00100000:1.0:1788845029.270452:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 925 00000100:00100000:1.0:1788845029.270463:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641527936:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845029.270521:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845029.270550:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287acc43 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f9 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.270585:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.270592:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845029.270860:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.270874:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.270935:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845029.271197:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967657 00000004:00010000:1.0:1788845029.271223:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 440975 -> 440975 00000004:00010000:1.0:1788845029.271227:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:1.0:1788845029.271229:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845028 -> 1788845026 00000004:00010000:1.0:1788845029.271245:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:1.0:1788845029.271247:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000004:00010000:1.0:1788845029.271253:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000796ce11c reply state 0000000083e6af98 transno 4294967658 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.271263:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845029.271359:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641527936:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 897us (1084us total) trans 4294967658 rc 0/0 00000100:00100000:1.0:1788845029.271366:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 925 00000020:00080000:1.0:1788845029.271371:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845029.278300:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.279111:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641528064 00000100:00100000:2.0:1788845029.279122:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641528064 00000100:00100000:2.0:1788845029.279172:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 926 00000100:00100000:2.0:1788845029.279182:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641528064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.279206:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.279225:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a851c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.279253:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.279262:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.279292:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.279322:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 14 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.279336:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660246016:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845029.279417:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788845029.280544:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.280558:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.285145:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.285155:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02756000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845029.285161:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.285170:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.285187:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.285191:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845029.285195:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.285202:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287acc66 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.285371:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660246016:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845029.285402:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.285418:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.285426:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.285432:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.285443:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1a00, rc 0) 00000100:00100000:2.0:1788845029.285504:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641528064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6322us (7213us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.285513:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 926 00000100:00100000:3.0:1788845029.286049:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.286169:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641528192 00000100:00100000:2.0:1788845029.286176:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641528192 00010000:00010000:2.0:1788845029.286203:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 15 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.286236:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 340 00000100:00100000:2.0:1788845029.286245:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641528192:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845029.286257:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845029.286283:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 15 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.286304:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 15 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.286328:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287acc2e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84eb expref: 15 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.286343:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.286350:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.286374:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.286385:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.286417:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.286430:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660246144:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845029.286496:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.290192:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660246144:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845029.290215:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845029.290260:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641528192:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4017us (4213us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.290270:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 340 00000100:00100000:3.0:1788845029.291469:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.291654:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641528320 00000100:00100000:1.0:1788845029.291663:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641528320 00000100:00100000:1.0:1788845029.291717:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 624 00000100:00100000:1.0:1788845029.291727:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641528320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.291746:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.291762:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8523 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.291794:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.291804:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 5) 00010000:00010000:1.0:1788845029.291808:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 5) 00010000:00010000:1.0:1788845029.291813:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 16 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.291848:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.291866:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 15 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.291879:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660246272:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845029.292140:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c92e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.304684:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660246272:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845029.304724:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 16 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.304746:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756000, rc 0) 00000100:00100000:1.0:1788845029.304807:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641528320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13080us (13343us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845029.304815:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 624 00000100:00100000:0.0:1788845029.305703:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.305783:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641528448 00000100:00100000:2.0:1788845029.305792:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641528448 00010000:00010000:2.0:1788845029.305827:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 16 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.305866:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 341 00000100:00100000:2.0:1788845029.305875:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008916c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641528448:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845029.305930:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845029.306111:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 16 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.306129:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 16 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.306153:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287acc35 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a84f2 expref: 16 pid: 6684 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.306182:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 5) 00010000:00010000:2.0:1788845029.306186:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 5) 00010000:00010000:2.0:1788845029.306194:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 15 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.306206:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 15 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.306243:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 15 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.306257:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660246400:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845029.306332:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.311751:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.311786:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660246400:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845029.311810:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845029.311864:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.312134:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641528576 00000100:00100000:1.0:1788845029.312142:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641528576 00000100:00100000:2.0:1788845029.312178:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008916c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641528448:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6304us (6477us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.312203:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 341 00002000:00010000:1.0:1788845029.312467:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845029.312472:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1180 with x1875739641528576 00000100:00100000:1.0:1788845029.312509:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 565 00000100:00100000:1.0:1788845029.312518:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641528576:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845029.312785:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 440975 -> 702376 00000004:00010000:1.0:1788845029.312790:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:1.0:1788845029.312792:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:1.0:1788845029.312794:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:1.0:1788845029.312797:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:1.0:1788845029.314825:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641528576:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 2294us (3069us total) trans 4294967659 rc 0/0 00000100:00100000:1.0:1788845029.314840:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 565 00002000:00010000:1.0:1788845029.314846:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845029.314850:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1180 with x1875739641528576 00000100:00100000:1.0:1788845029.314935:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641528704 00000100:00100000:1.0:1788845029.314972:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641528704 00002000:00010000:1.0:1788845029.315009:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845029.315013:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03610c450 with x1875739641528704 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845029.315023:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03610c450. 00000100:00100000:1.0:1788845029.315048:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 363 00000100:00100000:1.0:1788845029.315057:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03610c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641528704:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845029.315263:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1788845029.315266:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00002000:00010000:1.0:1788845029.315269:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845028 -> 1788845028 00002000:00010000:1.0:1788845029.315271:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00002000:00010000:1.0:1788845029.315273:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:1.0:1788845029.315319:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03610c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641528704:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 262us (3454us total) trans 4294967414 rc 0/0 00000100:00100000:1.0:1788845029.315326:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 363 00002000:00010000:1.0:1788845029.315329:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845029.315331:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03610c450 with x1875739641528704 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845029.315336:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03610c450. 00000100:00100000:1.0:1788845029.327929:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.331127:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641528832 00000100:00100000:2.0:1788845029.331143:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641528832 00002000:00100000:2.0:1788845029.331207:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ed180 x1875739641528832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.331218:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ed180 with x1875739641528832 00000100:00100000:2.0:1788845029.331253:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 566 00000100:00100000:2.0:1788845029.331264:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641528832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845029.331567:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641528832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 304us (3656us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845029.331575:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 566 00002000:00100000:2.0:1788845029.331581:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ed180 x1875739641528832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/4536 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.331589:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ed180 with x1875739641528832 00000100:00100000:2.0:1788845029.337157:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.337242:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641528960 00000100:00100000:3.0:1788845029.337254:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641528960 00002000:00100000:3.0:1788845029.337303:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c0a80 x1875739641528960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.337332:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0a80 with x1875739641528960 00000100:00100000:3.0:1788845029.337380:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 567 00000100:00100000:3.0:1788845029.337389:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641528960:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845029.337645:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641528960:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 255us (496us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845029.337652:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 567 00002000:00100000:3.0:1788845029.337659:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c0a80 x1875739641528960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/4536 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.337667:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0a80 with x1875739641528960 00000100:00100000:3.0:1788845029.343711:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.343834:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641529088 00000100:00100000:2.0:1788845029.343845:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641529088 00002000:00100000:2.0:1788845029.343919:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3b80 x1875739641529088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.343931:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3b80 with x1875739641529088 00000100:00100000:0.0:1788845029.344244:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 568 00000100:00100000:0.0:1788845029.344275:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641529088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845029.344560:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641529088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 289us (856us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845029.344577:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 568 00002000:00100000:0.0:1788845029.344585:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3b80 x1875739641529088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/4536 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845029.344593:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3b80 with x1875739641529088 00000100:00100000:1.0:1788845029.356085:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.356366:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641529216 00000100:00100000:3.0:1788845029.356376:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641529216 00002000:00100000:3.0:1788845029.356426:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360eca80 x1875739641529216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.356438:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360eca80 with x1875739641529216 00000100:00100000:3.0:1788845029.356478:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 569 00000100:00100000:3.0:1788845029.356489:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641529216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845029.356750:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641529216:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (672us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845029.356758:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 569 00002000:00100000:3.0:1788845029.356764:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360eca80 x1875739641529216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/4536 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.356772:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360eca80 with x1875739641529216 00000100:00100000:3.0:1788845029.362936:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.363493:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641529344 00000100:00100000:0.0:1788845029.363502:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641529344 00002000:00100000:0.0:1788845029.363572:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f0000 x1875739641529344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845029.363584:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f0000 with x1875739641529344 00000100:00100000:0.0:1788845029.363623:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 570 00000100:00100000:0.0:1788845029.363633:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641529344:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845029.363942:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641529344:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 309us (1062us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845029.363950:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 570 00002000:00100000:0.0:1788845029.364653:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f0000 x1875739641529344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/4536 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845029.364665:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f0000 with x1875739641529344 00000100:00100000:1.0:1788845029.367838:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.367947:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641529472 00000100:00100000:3.0:1788845029.367960:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641529472 00002000:00100000:3.0:1788845029.368037:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ec000 x1875739641529472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.368049:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ec000 with x1875739641529472 00000100:00100000:3.0:1788845029.368096:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 571 00000100:00100000:3.0:1788845029.368107:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641529472:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845029.368401:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641529472:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 294us (571us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845029.368409:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 571 00002000:00100000:3.0:1788845029.368417:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360ec000 x1875739641529472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/4536 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.368424:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360ec000 with x1875739641529472 00000100:00100000:1.0:1788845029.377180:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.377282:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641529600 00000100:00100000:2.0:1788845029.377294:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641529600 00002000:00100000:2.0:1788845029.377348:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360eea00 x1875739641529600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.377359:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360eea00 with x1875739641529600 00000100:00100000:2.0:1788845029.377417:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 572 00000100:00100000:2.0:1788845029.377427:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641529600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845029.377972:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641529600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 535us (790us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845029.377980:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 572 00002000:00100000:2.0:1788845029.377988:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360eea00 x1875739641529600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/4536 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.377996:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360eea00 with x1875739641529600 00000100:00100000:2.0:1788845029.394763:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.394930:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641529728 00000100:00100000:3.0:1788845029.394941:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641529728 00002000:00100000:3.0:1788845029.395001:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c0000 x1875739641529728/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.395012:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0000 with x1875739641529728 00000100:00100000:3.0:1788845029.395049:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 573 00000100:00100000:3.0:1788845029.395059:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:2370:x1875739641529728:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845029.403421:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 702376 -> 702376 00000004:00010000:3.0:1788845029.403431:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:3.0:1788845029.403433:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:3.0:1788845029.403436:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:3.0:1788845029.403448:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 968 00000100:00100000:3.0:1788845029.403775:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+26:2370:x1875739641529728:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 8712us (9016us total) trans 4294967660 rc 0/0 00000100:00100000:3.0:1788845029.403796:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 573 00002000:00100000:3.0:1788845029.403809:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c0000 x1875739641529728/t4294967660(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/448 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845029.403818:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0000 with x1875739641529728 00000100:00100000:0.0:1788845029.412924:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.413185:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641529856 00000100:00100000:1.0:1788845029.413195:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641529856 00000100:00100000:1.0:1788845029.413255:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 927 00000100:00100000:1.0:1788845029.413266:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+26:2368:x1875739641529856:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845029.497127:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967652 is committed 00000001:00080000:1.0:1788845029.497162:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967647 is committed 00000001:00080000:1.0:1788845029.497169:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967659 is committed 00000001:00080000:1.0:1788845029.497177:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967654 is committed 00000001:00080000:1.0:1788845029.497185:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967656 is committed 00000001:00080000:1.0:1788845029.497195:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967660 is committed 00000100:00100000:3.0:1788845029.497471:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641529856:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 84201us (84598us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845029.497483:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 927 00000100:00100000:2.0:1788845029.502757:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.502924:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641529984 00000100:00100000:0.0:1788845029.502934:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641529984 00000100:00100000:0.0:1788845029.502993:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 928 00000100:00100000:0.0:1788845029.503005:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641529984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.503046:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.503076:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acc7b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a852a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.503163:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acc82 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.503179:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acc82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.503190:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acc82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845029.503216:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 702376 00000004:00200000:0.0:1788845029.503221:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845029.503240:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acc82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.503267:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acc7b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a852a expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.503284:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acc82 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a852a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.503292:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.503295:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845029.503299:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:0.0:1788845029.503362:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641529984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 360us (615us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845029.503370:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 928 00000100:00100000:1.0:1788845029.514999:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.515043:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.515184:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641530112 00000100:00100000:0.0:1788845029.515195:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641530112 00000100:00100000:0.0:1788845029.515255:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 929 00000100:00100000:0.0:1788845029.515267:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641530112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.515290:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.515311:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acc89 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8531 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.515350:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.515406:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660246528:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.515502:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.515821:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641530240 00000100:00100000:2.0:1788845029.515826:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641530240 00000100:00100000:2.0:1788845029.515871:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 625 00000100:00100000:2.0:1788845029.515880:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641530240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.515934:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.515946:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287acc90 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8538 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.515986:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660246656:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.516017:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000004:00010000:0.0:1788845029.520456:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845029 00000100:00100000:0.0:1788845029.520494:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660246528:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845029.520521:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 702376 00010000:00010000:0.0:1788845029.520540:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acc89 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8531 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.520572:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.520576:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.520582:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acc89 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8531 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.520597:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:0.0:1788845029.520660:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641530112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5394us (5614us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.520686:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 929 00002000:00010000:2.0:1788845029.521186:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845028 -> 1788845029 00000100:00100000:2.0:1788845029.521204:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660246656:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845029.521230:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287acc90 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8538 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.521264:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287acc90 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8538 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.521275:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1000, rc 0) 00000100:00100000:2.0:1788845029.521313:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641530240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5435us (6325us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.521321:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 625 00000100:00100000:1.0:1788845029.531428:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.531478:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.531558:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641530368 00000100:00100000:0.0:1788845029.531581:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641530368 00000100:00100000:0.0:1788845029.531655:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 930 00000100:00100000:0.0:1788845029.531665:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641530368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.531687:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.531708:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acc97 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a853f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.531744:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.531795:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660246784:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.531927:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.532308:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641530496 00000100:00100000:2.0:1788845029.532314:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641530496 00000100:00100000:2.0:1788845029.532355:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 626 00000100:00100000:2.0:1788845029.532363:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641530496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.532373:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.532385:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287acc9e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8546 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.532424:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660246912:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.532473:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.537402:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660246912:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.537402:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660246784:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845029.537441:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 702376 00010000:00010000:2.0:1788845029.537455:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287acc9e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8546 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.537459:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acc97 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a853f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.537494:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.537498:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287acc9e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8546 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.537499:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.537508:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acc97 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a853f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.537525:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00010000:00010000:2.0:1788845029.537545:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1000, rc 0) 00000100:00100000:0.0:1788845029.537624:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641530368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5961us (6145us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.537636:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 930 00000100:00100000:2.0:1788845029.538191:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641530496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5827us (6774us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.538199:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 626 00000100:00100000:3.0:1788845029.544215:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.544406:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641530624 00000100:00100000:1.0:1788845029.544417:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641530624 00000100:00100000:1.0:1788845029.544538:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 931 00000100:00100000:1.0:1788845029.544550:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641530624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.544572:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.544594:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287acca5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a854d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.544680:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.544696:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.544709:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845029.544730:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 702376 00000004:00200000:1.0:1788845029.544733:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845029.544747:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.544803:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287acca5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.544821:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.544830:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.544833:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845029.544836:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757200, rc 0) 00000100:00100000:1.0:1788845029.544934:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641530624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 386us (727us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845029.544940:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 931 00000100:00100000:0.0:1788845029.549338:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.549409:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641530752 00000100:00100000:3.0:1788845029.549419:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641530752 00000100:00100000:3.0:1788845029.549489:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 932 00000100:00100000:3.0:1788845029.549500:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641530752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.549521:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.549543:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287accb3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8554 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.549578:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.549618:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660247040:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.549700:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.552163:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.552409:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660247040:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845029.552507:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 702376 00010000:00010000:3.0:1788845029.552547:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287accb3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8554 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.552568:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845029.552571:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845029.552576:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287accb3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8554 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.552591:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:3.0:1788845029.552701:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641530752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3202us (3370us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.552709:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 932 00000100:00100000:0.0:1788845029.552776:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641530880 00000100:00100000:0.0:1788845029.552792:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641530880 00000100:00100000:0.0:1788845029.552848:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 627 00000100:00100000:0.0:1788845029.552856:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641530880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.552870:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.552929:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287accba lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a855b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.552980:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660247168:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.553029:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.555939:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660247168:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845029.555983:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287accba lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a855b expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.556009:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287accba lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a855b expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.556024:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:0.0:1788845029.556085:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641530880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3227us (3929us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.556094:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 627 00000100:00100000:1.0:1788845029.558788:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.558851:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.558964:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641531008 00000100:00100000:2.0:1788845029.558973:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641531008 00000100:00100000:2.0:1788845029.559024:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 933 00000100:00100000:2.0:1788845029.559034:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641531008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.559053:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.559072:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287accc1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8562 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.559146:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641531136 00000100:00100000:0.0:1788845029.559164:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641531136 00000100:00100000:0.0:1788845029.559198:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 628 00000100:00100000:0.0:1788845029.559206:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641531136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.559215:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.559226:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287accc8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8569 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.559279:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660247296:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.559364:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.561158:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660247296:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845029.561190:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287accc8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8569 expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.561211:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287accc8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8569 expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.561221:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00010000:00010000:2.0:1788845029.561360:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.561392:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660247424:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.561452:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641531136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2245us (2672us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.561460:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 628 00000100:00100000:2.0:1788845029.561598:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.563488:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660247424:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.563511:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 702376 00010000:00010000:2.0:1788845029.563528:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287accc1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8562 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.563548:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.563561:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.563565:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287accc1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8562 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.563575:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:2.0:1788845029.563623:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641531008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4591us (4770us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.563632:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 933 00000100:00100000:1.0:1788845029.566294:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.566334:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641531392 00000100:00100000:0.0:1788845029.566344:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641531392 00000100:00100000:0.0:1788845029.566404:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 934 00000100:00100000:0.0:1788845029.566415:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641531392:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845029.566592:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845029.566641:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acc82 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a852a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.566660:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.566664:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845029.566941:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.566966:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.566977:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.567004:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.567027:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 14 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.567217:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660247552:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845029.567297:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.569429:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.569505:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641531520 00000100:00100000:2.0:1788845029.569512:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641531520 00010000:00010000:2.0:1788845029.569550:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 15 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.569578:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660247552:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845029.569594:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 342 00000100:00100000:2.0:1788845029.569603:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641531520:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845029.569610:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.569616:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845029.569622:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 15 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.569637:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 15 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.569658:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757200/0xb2187301287accac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a854d expref: 15 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.569673:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.569678:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.569702:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.569708:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.569728:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845029.569746:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.569764:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.569791:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641531520:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 179us (370us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.569797:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 342 00000001:00080000:0.0:1788845029.569875:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967661 00000004:00010000:0.0:1788845029.569982:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 702376 -> 702376 00000004:00010000:0.0:1788845029.569986:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:0.0:1788845029.569988:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845029 -> 1788845026 00000004:00010000:0.0:1788845029.569990:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:0.0:1788845029.569992:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 968 -> 712 00000004:00010000:0.0:1788845029.569998:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000065392b47 reply state 00000000e7a7bfd8 transno 4294967662 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.570006:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845029.570264:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641531392:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 3851us (3977us total) trans 4294967662 rc 0/0 00000100:00100000:0.0:1788845029.570271:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 934 00000020:00080000:0.0:1788845029.570276:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845029.572335:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.572351:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.572359:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.572363:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845029.572365:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191fa00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845029.572370:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.572400:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.572417:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.572420:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845029.572434:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.572453:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641531648 00000100:00100000:1.0:1788845029.572462:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641531648 00000100:00100000:1.0:1788845029.572517:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 935 00000100:00100000:1.0:1788845029.572528:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641531648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.572551:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.572566:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acccf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.572569:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8570 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.572596:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.572616:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.572649:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.572667:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 13 pid: 5841 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.572679:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660247680:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845029.572754:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.575267:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660247680:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845029.575296:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.575312:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.575314:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845029.575317:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.575324:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756400, rc 0) 00000100:00100000:1.0:1788845029.575373:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641531648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2848us (3014us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845029.575379:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 935 00000100:00100000:3.0:1788845029.576935:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.577679:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641531776 00000100:00100000:0.0:1788845029.577690:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641531776 00010000:00010000:0.0:1788845029.577725:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 14 pid: 5841 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.577758:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 343 00000100:00100000:0.0:1788845029.577767:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641531776:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845029.577781:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845029.577807:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 14 pid: 5841 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.577825:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 14 pid: 5841 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.577850:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1a00/0xb2187301287acc6d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a851c expref: 14 pid: 5841 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.577898:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.577915:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.577926:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.577939:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.577978:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.577991:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660247808:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845029.578857:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.581463:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.582182:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660247808:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845029.582204:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845029.582253:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641531776:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4487us (5325us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845029.582265:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 343 00000100:00100000:2.0:1788845029.582619:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641531904 00000100:00100000:2.0:1788845029.582628:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641531904 00000100:00100000:2.0:1788845029.582682:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 629 00000100:00100000:2.0:1788845029.582692:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641531904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.582709:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.582727:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8577 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.582756:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 16 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.582765:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 5) 00010000:00010000:2.0:1788845029.582770:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 5) 00010000:00010000:2.0:1788845029.582775:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.582806:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 16 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.582841:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 16 pid: 6685 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.582855:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660247936:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845029.582946:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.585692:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.585749:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660247936:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845029.585784:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.585805:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:2.0:1788845029.585858:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641531904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3168us (4405us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.585867:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 629 00000100:00100000:0.0:1788845029.586123:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641532032 00000100:00100000:0.0:1788845029.586131:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641532032 00010000:00010000:0.0:1788845029.586164:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 17 pid: 6685 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.586213:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 344 00000100:00100000:0.0:1788845029.586222:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641532032:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845029.586235:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845029.586257:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 17 pid: 6685 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.586274:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 17 pid: 6685 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.586297:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287acc74 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8523 expref: 17 pid: 6685 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.586311:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 5) 00010000:00010000:0.0:1788845029.586314:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 5) 00010000:00010000:0.0:1788845029.586321:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.586330:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.586360:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.586385:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660248064:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845029.586460:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.589242:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660248064:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845029.589271:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845029.589333:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641532032:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3111us (3648us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845029.589345:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 344 00000100:00100000:3.0:1788845029.591597:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.591692:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641532160 00000100:00100000:2.0:1788845029.591704:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641532160 00002000:00010000:2.0:1788845029.591759:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1592773->18446744073709551615) 00002000:00100000:2.0:1788845029.591764:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f2300 with x1875739641532160 ext(1592773->18446744073709551615) 00002000:00010000:2.0:1788845029.591776:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f2300. 00000100:00100000:2.0:1788845029.591811:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 364 00000100:00100000:2.0:1788845029.591822:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641532160:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845029.592091:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1592773 00002000:00010000:2.0:1788845029.592094:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00002000:00010000:2.0:1788845029.592096:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845029 -> 1788845028 00002000:00010000:2.0:1788845029.592109:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00002000:00010000:2.0:1788845029.592110:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:2.0:1788845029.592158:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641532160:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 337us (571us total) trans 4294967415 rc 0/0 00000100:00100000:2.0:1788845029.592166:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 364 00002000:00010000:2.0:1788845029.592170:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1592773->18446744073709551615) 00002000:00100000:2.0:1788845029.592172:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f2300 with x1875739641532160 ext(1592773->18446744073709551615) 00002000:00010000:2.0:1788845029.592177:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f2300. 00000100:00100000:3.0:1788845029.594273:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.594369:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641532288 00000100:00100000:2.0:1788845029.594378:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641532288 00000100:00100000:2.0:1788845029.594502:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 936 00000100:00100000:2.0:1788845029.594513:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641532288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.594536:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.594557:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287acce4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a857e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.594642:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287acceb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.594657:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287acceb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.594668:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287acceb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845029.594699:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1592773 00000004:00200000:2.0:1788845029.594703:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845029.594721:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287acceb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.594764:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287acce4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a857e expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.594851:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287acceb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a857e expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.594860:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.594862:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845029.594865:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:2.0:1788845029.595006:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641532288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 495us (740us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.595014:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 936 00000100:00100000:3.0:1788845029.597770:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.597823:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.597987:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641532416 00000100:00100000:2.0:1788845029.597997:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641532416 00000100:00100000:2.0:1788845029.598045:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 937 00000100:00100000:2.0:1788845029.598085:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641532416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.598104:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.598124:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287accf2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8585 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.598158:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.598194:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660248192:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.598261:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.598339:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641532544 00000100:00100000:0.0:1788845029.598343:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641532544 00000100:00100000:0.0:1788845029.598382:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 630 00000100:00100000:0.0:1788845029.598389:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641532544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.598398:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.598409:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287accf9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a858c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.598453:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660248320:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.598479:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00002000:00010000:0.0:1788845029.601571:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845028 -> 1788845029 00000100:00100000:0.0:1788845029.601602:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660248320:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845029.601638:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287accf9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a858c expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.601660:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287accf9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a858c expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.601687:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:0.0:1788845029.601743:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641532544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3355us (3923us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.601751:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 630 00000004:00010000:2.0:1788845029.601930:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845029 00000100:00100000:2.0:1788845029.601956:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660248192:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.601971:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 702376 00010000:00010000:2.0:1788845029.601983:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287accf2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8585 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.601996:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.601999:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.602003:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287accf2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8585 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.602012:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:2.0:1788845029.602036:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641532416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3954us (4275us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.602042:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 937 00000100:00100000:3.0:1788845029.605000:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.605194:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641532672 00000100:00100000:0.0:1788845029.605205:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641532672 00000100:00100000:0.0:1788845029.605275:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 938 00000100:00100000:0.0:1788845029.605285:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641532672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.605305:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.605326:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd00 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8593 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.605364:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.605402:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660248448:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.605633:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.607718:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.607820:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641532800 00000100:00100000:0.0:1788845029.607829:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641532800 00000100:00100000:0.0:1788845029.607923:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 631 00000100:00100000:0.0:1788845029.607934:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641532800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.607951:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.607968:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287acd07 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a859a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.608047:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660248576:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.608127:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.612984:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660248576:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.613043:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287acd07 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a859a expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.613068:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287acd07 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a859a expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.613081:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:1.0:1788845029.613141:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641532800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5208us (5432us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.613151:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 631 00000100:00100000:0.0:1788845029.613852:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660248448:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845029.613869:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 702376 00010000:00010000:0.0:1788845029.613934:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd00 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8593 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.613960:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.613964:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.613969:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd00 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8593 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.613982:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:0.0:1788845029.614036:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641532672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8752us (9044us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.614043:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 938 00000100:00100000:3.0:1788845029.618356:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.618446:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641532928 00000100:00100000:1.0:1788845029.618456:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641532928 00000100:00100000:1.0:1788845029.618522:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 939 00000100:00100000:1.0:1788845029.618533:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641532928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.618556:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.618577:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287acd0e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85a1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.618615:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.618656:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660248704:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.618741:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.621243:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.621422:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641533056 00000100:00100000:2.0:1788845029.621439:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641533056 00000100:00100000:2.0:1788845029.621498:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 632 00000100:00100000:2.0:1788845029.621509:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641533056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.621527:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.621548:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287acd15 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85a8 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.621603:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660248832:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.621678:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.624701:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660248704:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845029.624727:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 702376 00010000:00010000:1.0:1788845029.624759:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287acd0e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85a1 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.624792:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.624795:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845029.624801:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287acd0e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85a1 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.624815:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757c00, rc 0) 00000100:00100000:1.0:1788845029.624871:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641532928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6339us (6524us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.624880:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 939 00000100:00100000:2.0:1788845029.625696:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660248832:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845029.625721:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287acd15 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85a8 expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.625746:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000572b3fe0/0xb2187301287acd15 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85a8 expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.625758:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:2.0:1788845029.625806:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641533056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4300us (4572us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.625814:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 632 00000100:00100000:3.0:1788845029.630550:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.630594:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.630807:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641533184 00000100:00100000:2.0:1788845029.630818:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641533184 00000100:00100000:2.0:1788845029.630920:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 940 00000100:00100000:2.0:1788845029.630930:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641533184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.630951:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.630982:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287acd1c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85af expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.631020:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.631058:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660248960:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.631126:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.631863:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641533312 00000100:00100000:1.0:1788845029.631869:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641533312 00000100:00100000:1.0:1788845029.631969:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 633 00000100:00100000:1.0:1788845029.631979:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641533312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.631993:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.632007:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acd23 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85b6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.632046:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660249088:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.632098:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.635639:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660249088:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.635682:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acd23 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85b6 expref: 17 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.635728:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acd23 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85b6 expref: 17 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.635743:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757c00, rc 0) 00000100:00100000:1.0:1788845029.636615:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641533312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4631us (6013us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.636626:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 633 00000100:00100000:2.0:1788845029.638519:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660248960:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.638548:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 702376 00010000:00010000:2.0:1788845029.638571:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287acd1c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85af expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.638612:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.638615:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.638620:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287acd1c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85af expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.638634:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0600, rc 0) 00000100:00100000:2.0:1788845029.638698:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641533184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7769us (8157us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.638706:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 940 00000100:00100000:3.0:1788845029.642877:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.643110:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641533440 00000100:00100000:2.0:1788845029.643121:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641533440 00000100:00100000:2.0:1788845029.643178:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 941 00000100:00100000:2.0:1788845029.643188:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641533440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.643208:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.643228:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85bd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.643271:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.643283:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.643318:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.643349:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 11 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.643364:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660249216:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845029.643454:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.646963:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660249216:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845029.647005:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.647026:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.647029:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.647033:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.647043:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:2.0:1788845029.647106:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641533440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3919us (4238us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.647116:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 941 00000100:00100000:1.0:1788845029.649514:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.649770:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641533568 00000100:00100000:2.0:1788845029.649790:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641533568 00010000:00010000:2.0:1788845029.649832:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 12 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.649875:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 345 00000100:00100000:2.0:1788845029.649928:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641533568:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845029.649969:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845029.649999:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 12 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.650020:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 12 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.650041:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756400/0xb2187301287accd6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8570 expref: 12 pid: 5840 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.650057:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.650063:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.650074:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.650082:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.650113:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.650124:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660249344:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845029.650183:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.655295:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660249344:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845029.655338:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845029.655570:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641533568:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5684us (6063us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845029.655572:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.655582:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 345 00000100:00100000:0.0:1788845029.656578:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641533696 00000100:00100000:0.0:1788845029.656588:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641533696 00000100:00100000:0.0:1788845029.656685:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 634 00000100:00100000:0.0:1788845029.656694:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641533696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.656710:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.656724:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287acd31 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85c4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.656799:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660249472:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.656894:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.659436:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660249472:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845029.659476:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287acd31 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85c4 expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.659502:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287acd31 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85c4 expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.659515:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:0.0:1788845029.659609:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641533696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2914us (4035us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.659617:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 634 00000100:00100000:3.0:1788845029.662419:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.662749:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641533824 00000100:00100000:1.0:1788845029.662760:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641533824 00000100:00100000:1.0:1788845029.663096:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 635 00000100:00100000:1.0:1788845029.663104:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641533824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.663119:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.663133:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acd38 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85cb expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.663175:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660249600:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.663234:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.668291:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660249600:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.668337:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acd38 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85cb expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.668364:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000082fffaa8/0xb2187301287acd38 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85cb expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.668396:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757c00, rc 0) 00000100:00100000:1.0:1788845029.668518:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641533824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5413us (6107us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.668528:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 635 00000100:00100000:3.0:1788845029.671649:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.674334:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641533952 00000100:00100000:0.0:1788845029.674351:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641533952 00000100:00100000:0.0:1788845029.674437:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 942 00000100:00100000:0.0:1788845029.674449:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641533952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.674474:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.674499:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287acd3f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85d2 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.674602:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.674618:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.674628:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845029.674671:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1592773 00000004:00200000:0.0:1788845029.674676:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845029.674696:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.674723:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287acd3f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.674740:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.674750:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.674752:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845029.674756:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:0.0:1788845029.674831:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641533952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 383us (3191us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845029.674839:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 942 00000100:00100000:0.0:1788845029.678833:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.678945:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641534080 00000100:00100000:1.0:1788845029.679027:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641534080 00000100:00100000:1.0:1788845029.679094:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 943 00000100:00100000:1.0:1788845029.679105:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641534080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.679129:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.679150:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287acd4d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85d9 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.679187:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.679232:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660249728:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.679319:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000004:00010000:1.0:1788845029.687759:0:5840:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 702376 -> 991367 00000100:00100000:1.0:1788845029.687803:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660249728:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845029.687828:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:1.0:1788845029.687848:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287acd4d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d9 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.687869:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.687930:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845029.687937:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287acd4d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d9 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.688155:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757c00, rc 0) 00000100:00100000:1.0:1788845029.688256:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641534080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9151us (9431us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.688265:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 943 00000100:00100000:3.0:1788845029.691469:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.691670:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641534208 00000100:00100000:0.0:1788845029.691680:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641534208 00000100:00100000:0.0:1788845029.691747:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 944 00000100:00100000:0.0:1788845029.691758:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641534208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.691790:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.691811:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acd54 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85e0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.691849:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.692088:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660249856:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.692399:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.696131:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660249856:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.696161:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:2.0:1788845029.696180:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acd54 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85e0 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.696200:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.696203:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.696208:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287acd54 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85e0 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.696225:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:2.0:1788845029.696287:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641534208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4531us (4827us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.696296:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 944 00000100:00100000:3.0:1788845029.706315:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.706580:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641534336 00000100:00100000:0.0:1788845029.706664:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641534336 00002000:00100000:0.0:1788845029.706725:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c4b480 x1875739641534336/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845029.706736:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c4b480 with x1875739641534336 00000100:00100000:0.0:1788845029.706777:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 574 00000100:00100000:0.0:1788845029.707079:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641534336:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845029.721326:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641534336:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14528us (15012us total) trans 0 rc 704512/704512 00000100:00100000:2.0:1788845029.721341:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 574 00002000:00100000:2.0:1788845029.721353:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c4b480 x1875739641534336/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 488/440 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 704512/704512 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.721365:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c4b480 with x1875739641534336 00000100:00100000:3.0:1788845029.726168:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.726334:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641534464 00000100:00100000:2.0:1788845029.726345:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641534464 00000100:00100000:2.0:1788845029.726437:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 945 00000100:00100000:2.0:1788845029.726450:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641534464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.726473:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.726495:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acd5b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85e7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.726537:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.726578:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660249984:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.726665:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.730654:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660249984:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.730682:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:2.0:1788845029.730701:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acd5b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85e7 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.730722:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.730725:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.730729:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acd5b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85e7 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.730747:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:2.0:1788845029.730823:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641534464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4376us (4666us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.730834:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 945 00000100:00100000:0.0:1788845029.734711:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.734939:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641534592 00000100:00100000:2.0:1788845029.734949:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641534592 00000100:00100000:2.0:1788845029.735276:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 946 00000100:00100000:2.0:1788845029.735288:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641534592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.735310:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.735331:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acd62 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85ee expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.735370:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.735410:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660250112:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.735508:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.738962:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660250112:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.738994:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:2.0:1788845029.739027:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acd62 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85ee expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.739050:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.739054:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.739059:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acd62 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85ee expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.739074:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845029.739127:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641534592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3841us (4424us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.739135:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 946 00000100:00100000:1.0:1788845029.744459:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.744602:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641534720 00000100:00100000:3.0:1788845029.744614:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641534720 00000100:00100000:3.0:1788845029.744672:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 636 00000100:00100000:3.0:1788845029.744683:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641534720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.744703:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.744723:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287acd69 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85f5 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.744795:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660250240:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.744979:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.749961:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660250240:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845029.750008:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287acd69 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85f5 expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.750338:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287acd69 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85f5 expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.750353:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845029.750430:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641534720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5748us (5980us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.750456:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 636 00000100:00100000:1.0:1788845029.754550:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.754645:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641534848 00000100:00100000:2.0:1788845029.754655:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641534848 00000100:00100000:2.0:1788845029.754720:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 637 00000100:00100000:2.0:1788845029.754729:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641534848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.754747:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.754765:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acd70 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a85fc expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.754834:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660250368:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.754949:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.761674:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660250368:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845029.761715:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acd70 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85fc expref: 17 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.761741:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acd70 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85fc expref: 17 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.761756:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845029.761877:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641534848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7147us (7338us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.761942:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 637 00000100:00100000:0.0:1788845029.766273:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.766537:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641534976 00000100:00100000:3.0:1788845029.766547:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641534976 00000100:00100000:3.0:1788845029.766605:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 638 00000100:00100000:3.0:1788845029.766616:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641534976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.766634:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.766652:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287acd77 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8603 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.766707:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660250496:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.766795:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.770655:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660250496:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845029.770696:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287acd77 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8603 expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.770722:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287acd77 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8603 expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.770734:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845029.770806:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641534976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4190us (4541us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.770815:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 638 00000100:00100000:1.0:1788845029.774943:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.775139:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641535104 00000100:00100000:0.0:1788845029.775150:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641535104 00000100:00100000:0.0:1788845029.775217:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 639 00000100:00100000:0.0:1788845029.775228:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641535104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.775250:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.775270:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000073cade08/0xb2187301287acd7e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a860a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.775334:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660250624:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.775419:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.779215:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660250624:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845029.779256:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000073cade08/0xb2187301287acd7e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a860a expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.779295:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000073cade08/0xb2187301287acd7e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a860a expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.779308:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ec00, rc 0) 00000100:00100000:0.0:1788845029.779364:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641535104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4138us (4479us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.779373:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 639 00000100:00100000:1.0:1788845029.785590:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.785683:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641535232 00000100:00100000:3.0:1788845029.785695:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641535232 00000100:00100000:3.0:1788845029.785772:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 947 00000100:00100000:3.0:1788845029.785792:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641535232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.785812:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.785834:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8611 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.785868:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.785879:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.786171:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.786190:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 14 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.786204:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660250752:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845029.786286:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.790856:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660250752:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845029.791439:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.791460:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845029.791463:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845029.791482:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.791494:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:2.0:1788845029.791871:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641535232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6088us (6288us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.791926:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 947 00000100:00100000:1.0:1788845029.792505:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.794082:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641535360 00000100:00100000:3.0:1788845029.794096:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641535360 00002000:00100000:0.0:1788845029.794251:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358df80 x1875739641535360/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 8816/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845029.794263:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358df80 with x1875739641535360 00010000:00010000:0.0:1788845029.794273:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 15 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.794295:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 15 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845029.794304:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0358df80 00000100:00100000:0.0:1788845029.794383:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 23 00000100:00100000:0.0:1788845029.794394:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641535360:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845029.795085:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991367 -> 991367 00000004:00010000:0.0:1788845029.795090:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:0.0:1788845029.795092:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845029 -> 1788845026 00000004:00010000:0.0:1788845029.795095:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:0.0:1788845029.795102:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845029.795221:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641535360:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 828us (2717us total) trans 4294967663 rc 0/0 00000100:00100000:0.0:1788845029.795231:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 23 00002000:00100000:0.0:1788845029.795240:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358df80 x1875739641535360/t4294967663(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:135/0 lens 8816/448 e 0 to 0 dl 1788845040 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845029.795249:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358df80 with x1875739641535360 00010000:00010000:0.0:1788845029.795256:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 17 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.795273:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 17 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845029.795281:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0358df80 00000100:00100000:0.0:1788845029.800618:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.800730:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641535488 00000100:00100000:1.0:1788845029.800742:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641535488 00010000:00010000:1.0:1788845029.800855:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 17 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.800977:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 346 00000100:00100000:1.0:1788845029.800988:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641535488:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845029.801003:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845029.801030:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 17 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.801049:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 17 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.801072:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287acd2a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85bd expref: 17 pid: 5839 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.801104:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.801111:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.801122:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.801132:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.801169:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.801184:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660250880:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845029.801318:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.806208:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660250880:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845029.806238:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845029.806331:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641535488:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5343us (5721us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.806343:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 346 00000100:00100000:1.0:1788845029.809357:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.809414:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641535616 00000100:00100000:2.0:1788845029.809605:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641535616 00002000:00100000:2.0:1788845029.809664:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358dc00 x1875739641535616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.809676:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358dc00 with x1875739641535616 00000100:00100000:2.0:1788845029.809971:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 575 00000100:00100000:2.0:1788845029.809982:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641535616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845029.810047:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.810308:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641535744 00000100:00100000:1.0:1788845029.810314:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641535744 00002000:00100000:1.0:1788845029.810346:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1593343) req@ffff949f01c49f80 x1875739641535744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:135/0 lens 488/0 e 0 to 0 dl 1788845040 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845029.810357:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c49f80 with x1875739641535744 ext(1048576->1593343) 00002000:00010000:1.0:1788845029.810363:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c49f80 00000100:00100000:1.0:1788845029.810381:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 365 00000100:00100000:1.0:1788845029.810387:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641535744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845029.827720:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641535616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17727us (18356us total) trans 0 rc 995328/995328 00000100:00100000:2.0:1788845029.827737:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 575 00002000:00100000:2.0:1788845029.827753:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358dc00 x1875739641535616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:135/0 lens 488/440 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.827764:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358dc00 with x1875739641535616 00000100:00100000:3.0:1788845029.833754:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.834122:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.834254:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641535744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 23864us (24205us total) trans 0 rc 544768/544768 00000100:00100000:2.0:1788845029.834267:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 365 00002000:00100000:2.0:1788845029.834281:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1593343) req@ffff949f01c49f80 x1875739641535744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:135/0 lens 488/440 e 0 to 0 dl 1788845040 ref 1 fl Complete:/600/0 rc 544768/544768 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845029.834292:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c49f80 with x1875739641535744 ext(1048576->1593343) 00002000:00010000:2.0:1788845029.834298:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c49f80 00000100:00100000:1.0:1788845029.834397:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641536000 00000100:00100000:1.0:1788845029.834405:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641536000 00000100:00100000:1.0:1788845029.834465:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 640 00000100:00100000:1.0:1788845029.834475:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641536000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.834491:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.834511:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b801b586/0xb2187301287acd8c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a861f expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.834569:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660251008:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.834622:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.836106:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641535872 00000100:00100000:0.0:1788845029.836117:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641535872 00000100:00100000:3.0:1788845029.836488:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660251008:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845029.836524:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b801b586/0xb2187301287acd8c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a861f expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.836548:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b801b586/0xb2187301287acd8c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a861f expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.836559:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1a200, rc 0) 00000100:00100000:3.0:1788845029.836604:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641536000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2132us (2483us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.836610:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 640 00000100:00100000:0.0:1788845029.836876:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 948 00000100:00100000:0.0:1788845029.836922:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641535872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.836942:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.837278:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287acd93 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8618 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.837316:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287acd93 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8618 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845029.837330:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:0.0:1788845029.837356:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287acd93 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8618 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.837368:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.837371:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.837374:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fe00, rc 0) 00000100:00100000:0.0:1788845029.837413:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641535872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 531us (3669us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845029.837426:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 948 00000100:00100000:1.0:1788845029.840171:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.840291:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641536128 00000100:00100000:2.0:1788845029.840301:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641536128 00000100:00100000:2.0:1788845029.840380:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 641 00000100:00100000:2.0:1788845029.840390:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641536128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.840410:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.840566:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acd9a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8626 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.840628:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660251136:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.840708:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.842823:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660251136:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845029.842865:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acd9a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8626 expref: 17 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.843049:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acd9a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8626 expref: 17 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.843063:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845029.843121:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641536128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2732us (2956us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.843129:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 641 00000100:00100000:1.0:1788845029.846246:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.846580:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641536384 00000100:00100000:2.0:1788845029.846590:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641536384 00000100:00100000:2.0:1788845029.846686:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 949 00000100:00100000:2.0:1788845029.846697:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641536384:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845029.846753:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845029.846789:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287acceb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a857e expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.846822:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.846827:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845029.847186:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.847201:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.847210:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.847241:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.847377:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 12 pid: 11036 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.847393:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660251264:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845029.847485:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.850857:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660251264:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845029.850922:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.851284:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.851411:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641536512 00000100:00100000:0.0:1788845029.851419:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641536512 00010000:00010000:0.0:1788845029.851466:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 13 pid: 11036 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.851505:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 347 00000100:00100000:0.0:1788845029.851514:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641536512:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845029.851528:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845029.851535:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 13 pid: 11036 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.851553:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 13 pid: 11036 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.851575:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287acd46 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a85d2 expref: 13 pid: 11036 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.851590:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.851595:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.851620:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.851629:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.851661:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845029.851707:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641536512:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 194us (424us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845029.851715:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 347 00010000:00010000:3.0:1788845029.852659:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.852670:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845029.852784:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967664 00000004:00010000:3.0:1788845029.852845:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991367 -> 991367 00000004:00010000:3.0:1788845029.852849:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:3.0:1788845029.852852:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:3.0:1788845029.852854:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:3.0:1788845029.852857:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:3.0:1788845029.852865:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000be30e17e reply state 0000000069c6646e transno 4294967665 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.852874:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845029.853101:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641536384:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6404us (6862us total) trans 4294967665 rc 0/0 00000100:00100000:3.0:1788845029.853109:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 949 00000020:00080000:3.0:1788845029.853113:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845029.856814:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.857736:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641536640 00000100:00100000:2.0:1788845029.857749:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641536640 00000100:00100000:2.0:1788845029.857821:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 950 00000100:00100000:2.0:1788845029.857832:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641536640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.857869:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.857872:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845029.857946:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287acd93 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8618 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.858206:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.858212:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845029.858227:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a862d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.858252:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.858271:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.858306:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.858323:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 11 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.858337:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660251392:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845029.858524:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00010000:00010000:1.0:1788845029.859103:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.859114:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.859120:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.859126:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216e000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845029.859131:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.859139:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.859151:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.859163:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845029.859168:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.859173:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287acda1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.868842:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.868941:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660251392:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845029.868988:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.869009:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845029.869012:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845029.869017:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.869028:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:0.0:1788845029.870247:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641536768 00000100:00100000:0.0:1788845029.870270:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641536768 00010000:00010000:0.0:1788845029.870316:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 12 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.870359:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 348 00000100:00100000:0.0:1788845029.870369:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d48450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641536768:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845029.870383:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845029.870408:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 12 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.870434:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 12 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.870455:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287acd85 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8611 expref: 12 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.870487:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.870493:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.870502:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.870511:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.870551:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.870565:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660251520:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845029.870736:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641536640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12904us (13930us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845029.870745:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 950 00000100:00100000:0.0:1788845029.873698:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f480 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.877526:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.878064:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660251520:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845029.878090:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845029.878185:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d48450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641536768:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7817us (9355us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845029.878196:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 348 00000100:00100000:3.0:1788845029.878309:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641536896 00000100:00100000:3.0:1788845029.878328:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641536896 00000100:00100000:3.0:1788845029.878386:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 642 00000100:00100000:3.0:1788845029.878395:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641536896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.878414:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.878445:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8634 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.878477:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.878487:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(9 5) 00010000:00010000:3.0:1788845029.878491:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 5) 00010000:00010000:3.0:1788845029.878497:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.878530:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.878550:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 16 pid: 6684 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.878565:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660251648:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845029.878617:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.884210:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.884509:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660251648:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845029.884546:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.884579:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:0.0:1788845029.884745:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641537024 00000100:00100000:0.0:1788845029.884753:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641537024 00010000:00010000:0.0:1788845029.884849:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 17 pid: 6684 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.884936:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 349 00000100:00100000:0.0:1788845029.884945:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641537024:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845029.885094:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845029.885117:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 17 pid: 6684 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.885135:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 17 pid: 6684 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.885160:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287accdd lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8577 expref: 17 pid: 6684 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.885174:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 5) 00010000:00010000:0.0:1788845029.885178:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 5) 00010000:00010000:0.0:1788845029.885186:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.885196:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.885226:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.885251:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660251776:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845029.885561:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.888147:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660251776:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845029.888175:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845029.888221:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641537024:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3277us (4020us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845029.888232:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 349 00000100:00100000:2.0:1788845029.889228:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641536896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10826us (11704us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845029.889252:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 642 00000100:00100000:1.0:1788845029.893383:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.893667:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641537152 00000100:00100000:2.0:1788845029.893678:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641537152 00002000:00010000:2.0:1788845029.893718:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1237267->18446744073709551615) 00002000:00100000:2.0:1788845029.893723:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8dc00 with x1875739641537152 ext(1237267->18446744073709551615) 00002000:00010000:2.0:1788845029.893734:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8dc00. 00000100:00100000:2.0:1788845029.893769:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 366 00000100:00100000:2.0:1788845029.893788:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641537152:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845029.894260:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1592773 -> 1237267 00002000:00010000:2.0:1788845029.894264:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00002000:00010000:2.0:1788845029.894266:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845029 -> 1788845028 00002000:00010000:2.0:1788845029.894268:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00002000:00010000:2.0:1788845029.894270:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:2.0:1788845029.894326:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641537152:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 540us (953us total) trans 4294967416 rc 0/0 00000100:00100000:2.0:1788845029.894334:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 366 00002000:00010000:2.0:1788845029.894338:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1237267->18446744073709551615) 00002000:00100000:2.0:1788845029.894340:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8dc00 with x1875739641537152 ext(1237267->18446744073709551615) 00002000:00010000:2.0:1788845029.894345:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8dc00. 00000100:00100000:3.0:1788845029.897408:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.897562:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641537280 00000100:00100000:1.0:1788845029.897572:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641537280 00000100:00100000:1.0:1788845029.897634:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 951 00000100:00100000:1.0:1788845029.897645:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641537280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.897665:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.897687:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1a600/0xb2187301287acdb6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a863b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.897785:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.897800:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.897812:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845029.897840:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1237267 00000004:00200000:1.0:1788845029.897845:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845029.897865:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.897943:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1a600/0xb2187301287acdb6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.897978:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.897987:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.897990:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845029.897993:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b200, rc 0) 00000100:00100000:1.0:1788845029.898047:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641537280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 404us (648us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845029.898054:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 951 00000100:00100000:1.0:1788845029.900846:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.901009:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641537408 00000100:00100000:0.0:1788845029.901018:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641537408 00000100:00100000:0.0:1788845029.901079:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 952 00000100:00100000:0.0:1788845029.901090:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641537408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.901111:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.901130:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287acdc4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8642 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.901167:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.901211:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660251904:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.901298:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.903758:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.903956:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641537536 00000100:00100000:1.0:1788845029.903965:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641537536 00000100:00100000:1.0:1788845029.904026:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 643 00000100:00100000:1.0:1788845029.904036:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641537536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.904054:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.904072:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287acdcb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8649 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.904127:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660252032:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.904198:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ccc0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845029.904783:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845029 00000100:00100000:2.0:1788845029.904808:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660251904:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.904831:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:2.0:1788845029.904850:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287acdc4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8642 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.904923:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.904927:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.904933:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287acdc4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8642 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.904949:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:2.0:1788845029.904995:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641537408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3907us (4157us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.905003:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 952 00002000:00010000:1.0:1788845029.908806:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845028 -> 1788845029 00000100:00100000:1.0:1788845029.908841:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660252032:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.908922:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287acdcb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8649 expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.908968:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287acdcb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8649 expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.908982:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ba00, rc 0) 00000100:00100000:1.0:1788845029.909044:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641537536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5009us (5293us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.909052:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 643 00000100:00100000:3.0:1788845029.914871:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.915050:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641537664 00000100:00100000:1.0:1788845029.915061:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641537664 00000100:00100000:1.0:1788845029.915121:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 953 00000100:00100000:1.0:1788845029.915131:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641537664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.915151:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.915173:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ba00/0xb2187301287acdd2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8650 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.915210:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.915250:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660252160:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.915335:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ccc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.917741:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.917915:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641537792 00000100:00100000:3.0:1788845029.917924:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641537792 00000100:00100000:3.0:1788845029.917980:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 644 00000100:00100000:3.0:1788845029.917991:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641537792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.918010:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.918030:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287acdd9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8657 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.918088:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660252288:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.918169:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845029.918620:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660252160:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845029.918645:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:1.0:1788845029.918661:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ba00/0xb2187301287acdd2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8650 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.918701:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.918705:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845029.918710:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ba00/0xb2187301287acdd2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8650 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.918723:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ba00, rc 0) 00000100:00100000:1.0:1788845029.918760:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641537664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3631us (3898us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.918768:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 953 00000100:00100000:3.0:1788845029.923119:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660252288:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845029.923162:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287acdd9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8657 expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.923188:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287acdd9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8657 expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.923202:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:3.0:1788845029.923283:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641537792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5293us (5552us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.923309:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 644 00000100:00100000:0.0:1788845029.928477:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.928565:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641537920 00000100:00100000:3.0:1788845029.928576:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641537920 00000100:00100000:3.0:1788845029.928639:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 954 00000100:00100000:3.0:1788845029.928650:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641537920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.928671:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.928692:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287acde0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a865e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.928730:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.928773:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660252416:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.928868:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.931501:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.931640:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641538048 00000100:00100000:3.0:1788845029.931649:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641538048 00000100:00100000:3.0:1788845029.931713:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 645 00000100:00100000:3.0:1788845029.931723:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641538048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.931741:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.931758:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287acde7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8665 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.931820:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660252544:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.931932:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.932679:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660252416:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.932698:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:2.0:1788845029.932713:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287acde0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a865e expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.932730:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.932733:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.932738:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287acde0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a865e expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.932751:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:2.0:1788845029.932810:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641537920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4162us (4343us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.932817:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 954 00000100:00100000:3.0:1788845029.935964:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660252544:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845029.936019:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287acde7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8665 expref: 17 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.936043:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287acde7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8665 expref: 17 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.936057:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:3.0:1788845029.936117:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641538048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4394us (4623us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.936125:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 645 00000100:00100000:0.0:1788845029.939539:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.939583:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.939647:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641538304 00000100:00100000:3.0:1788845029.939654:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641538176 00000100:00100000:3.0:1788845029.939660:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641538176 00000100:00100000:2.0:1788845029.939660:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641538304 00000100:00100000:3.0:1788845029.939726:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 955 00000100:00100000:3.0:1788845029.939737:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641538176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.939827:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.939849:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287acdee lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a866c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.939921:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.939966:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660252672:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.940048:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.941144:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 646 00000100:00100000:2.0:1788845029.941159:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641538304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.941175:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.941203:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287acdf5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8673 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.941247:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660252800:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.941314:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.943498:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660252800:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845029.943539:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287acdf5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8673 expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.943567:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287acdf5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8673 expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.943582:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845029.943701:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641538304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2546us (4116us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.943709:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 646 00000100:00100000:0.0:1788845029.946680:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660252672:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845029.946710:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991367 00010000:00010000:0.0:1788845029.946730:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287acdee lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a866c expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.946752:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845029.946755:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845029.946761:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287acdee lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a866c expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.946787:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:0.0:1788845029.946941:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641538176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7205us (7412us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845029.946950:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 955 00000100:00100000:2.0:1788845029.950559:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.950623:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641538560 00000100:00100000:0.0:1788845029.950633:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641538560 00000100:00100000:0.0:1788845029.950693:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 956 00000100:00100000:0.0:1788845029.950704:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641538560:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845029.950992:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.951011:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.951020:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.951073:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.951092:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 11 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.951106:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f22a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660252928:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845029.951687:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845029.955770:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.955867:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f22a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660252928:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845029.955931:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.956136:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641538688 00000100:00100000:1.0:1788845029.956144:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641538688 00010000:00010000:1.0:1788845029.956178:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 12 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.956252:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 350 00000100:00100000:1.0:1788845029.956275:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641538688:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845029.956291:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845029.956299:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 12 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.956317:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 12 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.956343:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287acdbd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a863b expref: 12 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.956378:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.956383:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.956394:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.956402:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.956637:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845029.956694:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641538688:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 421us (933us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845029.956702:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 350 00010000:00010000:2.0:1788845029.956755:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.956763:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845029.956933:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967666 00000004:00010000:2.0:1788845029.956986:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991367 -> 991367 00000004:00010000:2.0:1788845029.956990:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:2.0:1788845029.956992:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845029 -> 1788845026 00000004:00010000:2.0:1788845029.956994:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:2.0:1788845029.956996:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:2.0:1788845029.957003:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004dc0e3ee reply state 00000000e19d9f63 transno 4294967667 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.957012:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845029.957079:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641538560:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6377us (6529us total) trans 4294967667 rc 0/0 00000100:00100000:2.0:1788845029.957086:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 956 00000020:00080000:2.0:1788845029.957090:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845029.958865:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.958914:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.958923:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.958929:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191e800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845029.958934:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.958941:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.958972:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.958975:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845029.958979:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.958985:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287acdfc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.960457:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.960496:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.960526:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641538944 00000100:00100000:2.0:1788845029.960536:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641538944 00002000:00010000:2.0:1788845029.960573:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845029.960578:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c2680 with x1875739641538944 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845029.960589:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c2680. 00000100:00100000:2.0:1788845029.960613:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 367 00000100:00100000:2.0:1788845029.960621:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641538944:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845029.960914:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1237267 -> 0 00002000:00010000:2.0:1788845029.960919:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00002000:00010000:2.0:1788845029.960922:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845029 -> 1788845028 00002000:00010000:2.0:1788845029.960924:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00002000:00010000:2.0:1788845029.960927:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:2.0:1788845029.960984:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641538944:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 363us (486us total) trans 4294967417 rc 0/0 00000100:00100000:2.0:1788845029.960991:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 367 00002000:00010000:2.0:1788845029.960994:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845029.961011:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c2680 with x1875739641538944 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845029.961017:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c2680. 00000100:00100000:3.0:1788845029.961338:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641538816 00000100:00100000:3.0:1788845029.961343:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641538816 00002000:00010000:3.0:1788845029.961368:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845029.961379:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c1500 with x1875739641538816 00000100:00100000:3.0:1788845029.961415:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 576 00000100:00100000:3.0:1788845029.961423:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641538816:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845029.961638:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991367 -> 390394 00000004:00010000:3.0:1788845029.961642:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845029 00000004:00010000:3.0:1788845029.961644:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:3.0:1788845029.961646:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845029 00000004:00010000:3.0:1788845029.961648:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845029.961692:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641538816:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 269us (1241us total) trans 4294967668 rc 0/0 00000100:00100000:3.0:1788845029.961699:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 576 00002000:00010000:3.0:1788845029.961702:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845029.961705:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c1500 with x1875739641538816 00000100:00100000:2.0:1788845029.965665:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.965763:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641539072 00000100:00100000:1.0:1788845029.965774:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641539072 00000100:00100000:1.0:1788845029.965982:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 957 00000100:00100000:1.0:1788845029.965994:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641539072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.966015:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.966035:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ace03 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a867a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.966120:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace0a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.966135:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.966164:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845029.966192:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 390394 00000004:00200000:1.0:1788845029.966196:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845029.966215:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.966241:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ace03 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a867a expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.966259:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace0a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a867a expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.966268:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.966270:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845029.966274:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ae00, rc 0) 00000100:00100000:1.0:1788845029.966379:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641539072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 386us (722us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845029.966386:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 957 00010000:00010000:3.0:1788845029.970029:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845029.972023:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845029.972164:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.972232:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641539328 00000100:00100000:1.0:1788845029.972242:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641539328 00000100:00100000:1.0:1788845029.972302:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 647 00000100:00100000:1.0:1788845029.972313:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641539328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845029.972332:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845029.972350:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287ace11 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8688 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845029.972438:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660253056:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845029.972524:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ccc0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845029.974735:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845028 -> 1788845029 00000100:00100000:1.0:1788845029.974769:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660253056:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845029.974815:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287ace11 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8688 expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.974837:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287ace11 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8688 expref: 17 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.974848:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ac00, rc 0) 00000100:00100000:1.0:1788845029.974939:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641539328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2628us (2774us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.974948:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 647 00000100:00100000:2.0:1788845029.975688:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641539200 00000100:00100000:2.0:1788845029.975697:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641539200 00000100:00100000:2.0:1788845029.975740:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 958 00000100:00100000:2.0:1788845029.975749:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641539200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.975767:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.975794:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace18 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8681 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.975827:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.975865:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660253184:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845029.975975:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845029.979412:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845029 00000100:00100000:2.0:1788845029.979446:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660253184:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845029.979473:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 390394 00010000:00010000:2.0:1788845029.979492:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace18 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8681 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.979513:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.979517:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845029.979522:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8681 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.979563:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845029.979628:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641539200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3878us (7612us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845029.979636:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 958 00000100:00100000:1.0:1788845029.982739:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845029.982797:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845029.982937:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641539584 00000100:00100000:3.0:1788845029.982946:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641539584 00000100:00100000:3.0:1788845029.983001:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 648 00000100:00100000:3.0:1788845029.983012:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641539584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845029.983031:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845029.983049:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace1f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8696 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845029.983109:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660253312:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845029.983189:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845029.983337:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641539456 00000100:00100000:0.0:1788845029.983342:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641539456 00000100:00100000:0.0:1788845029.983383:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 959 00000100:00100000:0.0:1788845029.983392:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641539456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845029.983401:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845029.983415:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ace26 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a868f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845029.983446:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.983469:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660253440:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845029.983512:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845029.986975:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660253312:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845029.987033:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace1f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8696 expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.987059:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace1f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8696 expref: 17 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845029.987072:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:3.0:1788845029.987133:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641539584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4123us (4335us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845029.987143:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 648 00000100:00100000:1.0:1788845029.987709:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660253440:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845029.987729:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 390394 00010000:00010000:1.0:1788845029.987743:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ace26 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a868f expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.987757:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.987760:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845029.987783:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ace26 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a868f expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.987796:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:1.0:1788845029.987839:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641539456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4449us (5111us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845029.987846:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 959 00000100:00100000:0.0:1788845029.993343:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.993457:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641539840 00000100:00100000:2.0:1788845029.993468:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641539840 00000100:00100000:2.0:1788845029.993533:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 960 00000100:00100000:2.0:1788845029.993545:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641539840:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845029.993608:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845029.993636:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a867a expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.993655:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845029.993660:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845029.993937:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.993951:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.993962:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845029.994020:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967669 00000004:00010000:2.0:1788845029.994045:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 390394 -> 390394 00000004:00010000:2.0:1788845029.994049:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845029 -> 1788845030 00000004:00010000:2.0:1788845029.994052:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845029 -> 1788845026 00000004:00010000:2.0:1788845029.994066:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845029 -> 1788845030 00000004:00010000:2.0:1788845029.994069:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:2.0:1788845029.994075:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000032d53d7c reply state 0000000059b6c5cf transno 4294967670 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.994083:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845029.994166:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641539840:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 624us (832us total) trans 4294967670 rc 0/0 00000100:00100000:2.0:1788845029.994174:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 960 00000020:00080000:2.0:1788845029.994179:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845029.996344:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.996362:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.996370:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.996377:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216fe00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845029.996382:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.996389:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.996404:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845029.996407:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845029.996411:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845029.996427:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ace2d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845029.998377:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845029.998564:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641539968 00000100:00100000:2.0:1788845029.998574:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641539968 00000100:00100000:2.0:1788845029.998635:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 961 00000100:00100000:2.0:1788845029.998646:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641539968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845029.998673:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845029.998696:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a869d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.998727:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.998863:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.998953:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845029.998982:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 20 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845029.999084:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660253568:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845029.999163:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.005271:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.006213:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641540096 00000100:00100000:0.0:1788845030.006227:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641540096 00010000:00010000:0.0:1788845030.006284:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 21 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.006333:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 351 00000100:00100000:0.0:1788845030.006344:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641540096:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845030.006360:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845030.006386:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 21 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.006405:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 21 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.006561:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.006570:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.006582:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.006591:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.006648:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.006662:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660253696:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845030.006788:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788845030.006924:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287acda8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a862d expref: 21 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.006941:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660253568:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845030.006966:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.006977:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845030.006980:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845030.006982:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845030.007013:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641539968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8369us (8646us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.007021:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 961 00000100:00100000:0.0:1788845030.011475:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.011550:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641540224 00000100:00100000:3.0:1788845030.011560:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641540224 00000100:00100000:3.0:1788845030.011618:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 649 00000100:00100000:3.0:1788845030.011629:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641540224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845030.011648:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845030.011666:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86a4 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.011698:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 18 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.011707:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 5) 00010000:00010000:3.0:1788845030.011711:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 5) 00010000:00010000:3.0:1788845030.011716:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 17 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.011778:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 18 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.011800:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 18 pid: 6682 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845030.011814:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660253824:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845030.011934:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845030.014709:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.014826:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660253824:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845030.014861:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 17 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.014920:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:2.0:1788845030.014975:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641540224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3347us (3510us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.014984:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 649 00000100:00100000:0.0:1788845030.015082:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641540352 00000100:00100000:0.0:1788845030.015089:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641540352 00010000:00010000:0.0:1788845030.015118:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 19 pid: 6682 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.015156:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 352 00000100:00100000:0.0:1788845030.015165:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641540352:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845030.015179:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845030.015201:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 19 pid: 6682 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.015217:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 19 pid: 6682 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.015240:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287acdaf lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8634 expref: 19 pid: 6682 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.015253:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 5) 00010000:00010000:0.0:1788845030.015265:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 5) 00010000:00010000:0.0:1788845030.015272:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.015283:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.015313:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.015326:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660253952:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845030.015375:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.018361:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f22300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660253952:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845030.018391:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845030.018465:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641540352:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3300us (3765us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.018476:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 352 00000100:00100000:1.0:1788845030.022648:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660253696:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845030.022677:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845030.022966:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.022998:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641540480 00000100:00100000:3.0:1788845030.023005:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641540480 00002000:00010000:3.0:1788845030.023042:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1049696->18446744073709551615) 00002000:00100000:3.0:1788845030.023047:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01f98700 with x1875739641540480 ext(1049696->18446744073709551615) 00002000:00010000:3.0:1788845030.023057:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01f98700. 00000100:00100000:3.0:1788845030.023082:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 368 00000100:00100000:3.0:1788845030.023091:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641540480:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845030.023312:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1049696 00002000:00010000:3.0:1788845030.023316:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845029 -> 1788845030 00002000:00010000:3.0:1788845030.023319:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845029 -> 1788845028 00002000:00010000:3.0:1788845030.023321:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845029 -> 1788845030 00002000:00010000:3.0:1788845030.023323:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 129 00000100:00100000:3.0:1788845030.023370:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641540480:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 279us (405us total) trans 4294967418 rc 0/0 00000100:00100000:3.0:1788845030.023391:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 368 00002000:00010000:3.0:1788845030.023395:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1049696->18446744073709551615) 00002000:00100000:3.0:1788845030.023398:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01f98700 with x1875739641540480 ext(1049696->18446744073709551615) 00002000:00010000:3.0:1788845030.023402:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01f98700. 00000100:00100000:3.0:1788845030.026853:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.027035:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641540608 00000100:00100000:2.0:1788845030.027045:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641540608 00002000:00100000:2.0:1788845030.027091:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9ad80 x1875739641540608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1788845030.027092:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641540096:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 20749us (21841us total) trans 0 rc 0/0 00002000:00100000:2.0:1788845030.027101:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9ad80 with x1875739641540608 00000100:00100000:0.0:1788845030.027104:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 351 00000100:00100000:2.0:1788845030.027128:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 577 00000100:00100000:2.0:1788845030.027137:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641540608:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.027352:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641540608:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 215us (508us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.027359:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 577 00002000:00100000:2.0:1788845030.027365:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9ad80 x1875739641540608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/440 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.027372:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9ad80 with x1875739641540608 00000100:00100000:2.0:1788845030.032323:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.032413:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641540736 00000100:00100000:3.0:1788845030.032434:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641540736 00002000:00100000:3.0:1788845030.032484:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c3100 x1875739641540736/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.032496:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c3100 with x1875739641540736 00000100:00100000:3.0:1788845030.032535:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 578 00000100:00100000:3.0:1788845030.032545:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641540736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845030.033046:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641540736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 501us (730us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845030.033054:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 578 00002000:00100000:3.0:1788845030.033062:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c3100 x1875739641540736/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/440 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.033071:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c3100 with x1875739641540736 00000100:00100000:1.0:1788845030.036444:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.036580:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641540864 00000100:00100000:2.0:1788845030.036590:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641540864 00002000:00100000:2.0:1788845030.036636:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4a80 x1875739641540864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.036648:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4a80 with x1875739641540864 00000100:00100000:2.0:1788845030.036692:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 579 00000100:00100000:2.0:1788845030.036701:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641540864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.037119:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641540864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 416us (684us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.037128:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 579 00002000:00100000:2.0:1788845030.037136:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4a80 x1875739641540864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/440 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.037159:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4a80 with x1875739641540864 00000100:00100000:2.0:1788845030.040763:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.040858:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641540992 00000100:00100000:0.0:1788845030.040872:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641540992 00002000:00100000:0.0:1788845030.040957:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c2a00 x1875739641540992/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.040969:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c2a00 with x1875739641540992 00000100:00100000:0.0:1788845030.041009:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 580 00000100:00100000:0.0:1788845030.041032:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641540992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.041270:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641540992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 239us (516us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.041278:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 580 00002000:00100000:0.0:1788845030.041285:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c2a00 x1875739641540992/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/440 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.041292:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c2a00 with x1875739641540992 00000100:00100000:2.0:1788845030.044809:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.045028:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641541120 00000100:00100000:0.0:1788845030.045038:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641541120 00002000:00100000:0.0:1788845030.045087:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c0700 x1875739641541120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.045099:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0700 with x1875739641541120 00000100:00100000:0.0:1788845030.045136:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 581 00000100:00100000:0.0:1788845030.045147:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641541120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.045430:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641541120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 283us (631us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.045439:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 581 00002000:00100000:0.0:1788845030.045447:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c0700 x1875739641541120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/440 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.045455:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c0700 with x1875739641541120 00000100:00100000:1.0:1788845030.048936:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.049031:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641541248 00000100:00100000:2.0:1788845030.049041:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641541248 00002000:00100000:2.0:1788845030.049091:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1052671) req@ffff94a0360b5c00 x1875739641541248/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.049101:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5c00 with x1875739641541248 ext(1048576->1052671) 00002000:00010000:2.0:1788845030.049107:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5c00 00000100:00100000:2.0:1788845030.049142:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 369 00000100:00100000:2.0:1788845030.049153:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641541248:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.049385:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641541248:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 232us (510us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.049394:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 369 00002000:00100000:2.0:1788845030.049401:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1052671) req@ffff94a0360b5c00 x1875739641541248/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.049410:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5c00 with x1875739641541248 ext(1048576->1052671) 00002000:00010000:2.0:1788845030.049541:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5c00 00000100:00100000:3.0:1788845030.053133:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.053540:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641541376 00000100:00100000:0.0:1788845030.053552:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641541376 00002000:00100000:0.0:1788845030.053613:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9aa00 x1875739641541376/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.053624:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9aa00 with x1875739641541376 00000100:00100000:0.0:1788845030.053686:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 582 00000100:00100000:0.0:1788845030.053697:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641541376:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845030.055593:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.055693:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641541504 00000100:00100000:2.0:1788845030.055702:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641541504 00002000:00100000:2.0:1788845030.055748:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1049695) req@ffff94a034f22300 x1875739641541504/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 1608/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.055759:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f22300 with x1875739641541504 ext(1048576->1049695) 00002000:00010000:2.0:1788845030.055764:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f22300 00000100:00100000:2.0:1788845030.055808:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 370 00000100:00100000:2.0:1788845030.055817:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2367:x1875739641541504:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845030.056298:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2367:x1875739641541504:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 480us (712us total) trans 4294967419 rc 0/0 00000100:00100000:2.0:1788845030.056306:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 370 00002000:00100000:2.0:1788845030.056314:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1049695) req@ffff94a034f22300 x1875739641541504/t4294967419(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 1608/448 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.056323:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f22300 with x1875739641541504 ext(1048576->1049695) 00002000:00010000:2.0:1788845030.056327:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f22300 00000004:00010000:2.0:1788845030.062227:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 390394 -> 1048576 00000004:00010000:2.0:1788845030.062239:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845030 -> 1788845030 00000004:00010000:2.0:1788845030.062242:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845026 -> 1788845026 00000004:00010000:2.0:1788845030.062244:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845030 -> 1788845030 00000004:00010000:2.0:1788845030.062246:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:2.0:1788845030.062325:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641541376:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 8625us (9197us total) trans 4294967671 rc 0/0 00000100:00100000:2.0:1788845030.062337:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 582 00002000:00100000:2.0:1788845030.062349:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9aa00 x1875739641541376/t4294967671(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/448 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.062359:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9aa00 with x1875739641541376 00000100:00100000:0.0:1788845030.067194:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.067342:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641541632 00000100:00100000:2.0:1788845030.067354:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641541632 00000100:00100000:2.0:1788845030.067425:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 962 00000100:00100000:2.0:1788845030.067435:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641541632:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:2.0:1788845030.074519:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.074603:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641541760 00000100:00100000:1.0:1788845030.074612:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641541760 00000100:00100000:1.0:1788845030.074669:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 650 00000100:00100000:1.0:1788845030.074680:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641541760:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:1.0:1788845030.107142:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641541760:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 32451us (32618us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845030.107156:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 650 00000001:00080000:1.0:1788845030.107296:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967408 is committed 00000001:00080000:1.0:1788845030.107309:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967403 is committed 00000001:00080000:1.0:1788845030.107317:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967414 is committed 00000001:00080000:1.0:1788845030.107321:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967413 is committed 00000001:00080000:1.0:1788845030.107353:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967419 is committed 00000001:00080000:1.0:1788845030.107361:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967417 is committed 00000100:00100000:0.0:1788845030.111699:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641541632:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 44254us (44516us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.111713:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 962 00000001:00080000:0.0:1788845030.112018:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967666 is committed 00000001:00080000:0.0:1788845030.112033:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967662 is committed 00000001:00080000:0.0:1788845030.112042:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967671 is committed 00000001:00080000:0.0:1788845030.112052:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967667 is committed 00000001:00080000:0.0:1788845030.112059:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967668 is committed 00000100:00100000:2.0:1788845030.116212:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.116371:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641541888 00000100:00100000:1.0:1788845030.116383:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641541888 00000100:00100000:1.0:1788845030.116458:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 963 00000100:00100000:1.0:1788845030.116467:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641541888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845030.116488:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845030.116508:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ace42 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86ab expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.116599:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.116634:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.116645:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845030.116673:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1049696 00000004:00200000:1.0:1788845030.116677:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845030.116696:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.116722:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ace42 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.116739:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.116748:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845030.116751:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845030.116755:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ae00, rc 0) 00000100:00100000:1.0:1788845030.116877:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641541888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 410us (673us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845030.117079:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 963 00000100:00100000:2.0:1788845030.123751:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.123880:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641542016 00000100:00100000:3.0:1788845030.124037:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641542016 00000100:00100000:0.0:1788845030.124222:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 964 00000100:00100000:0.0:1788845030.124235:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641542016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.124273:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.124297:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ace50 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86b2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.124394:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287ace57 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.124410:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287ace57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.124437:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287ace57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845030.124465:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1049696 00000004:00200000:0.0:1788845030.124485:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845030.124506:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287ace57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.124534:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ace50 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86b2 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.124553:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287ace57 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86b2 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.124562:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.124566:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845030.124569:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ea00, rc 0) 00000100:00100000:0.0:1788845030.124640:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641542016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 408us (899us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.124648:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 964 00000100:00100000:0.0:1788845030.127708:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.127815:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.128147:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641542144 00000100:00100000:1.0:1788845030.128157:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641542144 00000100:00100000:1.0:1788845030.128269:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 965 00000100:00100000:1.0:1788845030.128281:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641542144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845030.128302:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845030.128334:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ace5e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86b9 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.128372:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845030.128411:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b5500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660254080:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845030.128517:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.129188:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641542272 00000100:00100000:2.0:1788845030.129196:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641542272 00000100:00100000:2.0:1788845030.129344:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 651 00000100:00100000:2.0:1788845030.129353:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641542272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.129367:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.129380:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ace65 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86c0 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.129434:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660254208:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.129492:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c840 going to sleep for 11 seconds 00000004:00010000:3.0:1788845030.131125:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845030 00000100:00100000:3.0:1788845030.131154:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b5500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660254080:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845030.131250:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845030.131289:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ace5e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86b9 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.131309:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845030.131313:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845030.131318:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ace5e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86b9 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.131332:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b200, rc 0) 00000100:00100000:3.0:1788845030.131559:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641542144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3279us (3861us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845030.131567:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 965 00002000:00010000:2.0:1788845030.137254:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845028 -> 1788845030 00000100:00100000:2.0:1788845030.137295:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660254208:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845030.137340:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ace65 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86c0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.137368:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ace65 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86c0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.137384:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845030.137640:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641542272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8286us (9823us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845030.137649:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 651 00000100:00100000:1.0:1788845030.140848:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.141082:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.141156:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641542400 00000100:00100000:0.0:1788845030.141166:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641542400 00000100:00100000:0.0:1788845030.141239:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 966 00000100:00100000:0.0:1788845030.141250:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641542400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.141285:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.141306:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ace6c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86c7 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.141347:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.141410:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660254336:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.141516:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.142751:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641542528 00000100:00100000:2.0:1788845030.142761:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641542528 00000100:00100000:2.0:1788845030.142978:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 652 00000100:00100000:2.0:1788845030.142989:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641542528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.143005:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.143022:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ace73 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86ce expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.143072:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660254464:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.143136:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.146757:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660254336:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.146786:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.146806:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ace6c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86c7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.146828:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.146831:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.146836:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ace6c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86c7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.146852:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:0.0:1788845030.147403:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641542400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6151us (6317us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.147421:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 966 00000100:00100000:3.0:1788845030.149110:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660254464:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845030.149150:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ace73 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ce expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.149186:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ace73 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ce expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.149202:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845030.149269:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641542528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6282us (8431us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845030.149277:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 652 00000100:00100000:1.0:1788845030.153027:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.153104:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.153241:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641542784 00000100:00100000:0.0:1788845030.153252:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641542784 00000100:00100000:0.0:1788845030.153327:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 653 00000100:00100000:0.0:1788845030.153339:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641542784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.153359:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.153379:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ace7a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86dc expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.153448:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660254592:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.153552:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845030.155595:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641542656 00000100:00100000:3.0:1788845030.155610:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641542656 00000100:00100000:0.0:1788845030.156030:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660254592:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845030.156070:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ace7a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86dc expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.156097:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ace7a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86dc expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.156109:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:0.0:1788845030.156172:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641542784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2835us (3066us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.156190:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 653 00000100:00100000:2.0:1788845030.157226:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 967 00000100:00100000:2.0:1788845030.157239:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641542656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.157268:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.157289:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ace81 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86d5 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.157325:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.157367:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660254720:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.157451:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.161383:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660254720:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845030.161551:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845030.161576:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ace81 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86d5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.161621:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845030.161625:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845030.161630:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ace81 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86d5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.161647:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845030.161764:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641542656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4527us (8746us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845030.161782:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 967 00000100:00100000:1.0:1788845030.165725:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.165831:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641542912 00000100:00100000:0.0:1788845030.165840:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641542912 00000100:00100000:0.0:1788845030.165941:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 968 00000100:00100000:0.0:1788845030.165952:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641542912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.165977:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.166001:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86e3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.166031:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.166040:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.166095:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.166114:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 10 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.166128:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660254848:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845030.166401:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.169749:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.169851:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641543040 00000100:00100000:2.0:1788845030.169862:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641543040 00010000:00010000:2.0:1788845030.169938:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 11 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.169975:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 353 00000100:00100000:2.0:1788845030.169985:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641543040:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845030.170001:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845030.170029:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 11 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.170045:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 11 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.170068:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845030.170073:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.170101:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.170109:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.170147:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.170161:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660254976:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845030.170269:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00010000:00010000:0.0:1788845030.173348:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ace34 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a869d expref: 11 pid: 5840 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.173380:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660254848:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845030.173426:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.173444:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.173447:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.173449:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:0.0:1788845030.173580:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641542912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7630us (7866us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.173588:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 968 00000100:00100000:2.0:1788845030.176693:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660254976:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845030.176720:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845030.176786:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641543040:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6803us (7047us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.176798:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 353 00000100:00100000:1.0:1788845030.179848:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.180129:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641543168 00000100:00100000:0.0:1788845030.180139:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641543168 00002000:00100000:0.0:1788845030.180309:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5f80 x1875739641543168/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 504/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.180343:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5f80 with x1875739641543168 00000100:00100000:0.0:1788845030.180431:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 583 00000100:00100000:0.0:1788845030.180443:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641543168:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.185092:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641543168:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4641us (5243us total) trans 0 rc 57344/57344 00000100:00100000:0.0:1788845030.185107:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 583 00002000:00100000:0.0:1788845030.185121:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5f80 x1875739641543168/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 504/440 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.185132:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5f80 with x1875739641543168 00000100:00100000:2.0:1788845030.188616:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.188737:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641543296 00000100:00100000:3.0:1788845030.188748:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641543296 00002000:00100000:3.0:1788845030.188823:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03411c00 x1875739641543296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.188836:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03411c00 with x1875739641543296 00000100:00100000:3.0:1788845030.188949:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 584 00000100:00100000:3.0:1788845030.188960:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641543296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845030.189273:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641543296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 313us (666us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845030.189282:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 584 00002000:00100000:3.0:1788845030.189290:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03411c00 x1875739641543296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.189299:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03411c00 with x1875739641543296 00000100:00100000:0.0:1788845030.193652:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.193845:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641543424 00000100:00100000:2.0:1788845030.193855:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641543424 00002000:00100000:2.0:1788845030.193933:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f20380 x1875739641543424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.193944:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20380 with x1875739641543424 00000100:00100000:2.0:1788845030.193982:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 585 00000100:00100000:2.0:1788845030.193992:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641543424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.194249:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641543424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 257us (605us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.194257:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 585 00002000:00100000:2.0:1788845030.194264:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f20380 x1875739641543424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.194273:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20380 with x1875739641543424 00000100:00100000:1.0:1788845030.199266:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.199372:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641543552 00000100:00100000:0.0:1788845030.199382:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641543552 00002000:00100000:0.0:1788845030.199446:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5500 x1875739641543552/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.199458:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5500 with x1875739641543552 00000100:00100000:0.0:1788845030.199500:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 586 00000100:00100000:0.0:1788845030.199512:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641543552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.199819:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641543552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 308us (563us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.199827:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 586 00002000:00100000:0.0:1788845030.199835:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5500 x1875739641543552/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.199842:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5500 with x1875739641543552 00000100:00100000:3.0:1788845030.203566:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.203736:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641543680 00000100:00100000:2.0:1788845030.203746:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641543680 00002000:00100000:2.0:1788845030.203799:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f98a80 x1875739641543680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.203809:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98a80 with x1875739641543680 00000100:00100000:2.0:1788845030.203850:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 587 00000100:00100000:2.0:1788845030.203861:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641543680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.204152:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641543680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 291us (596us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.204160:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 587 00002000:00100000:2.0:1788845030.204168:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f98a80 x1875739641543680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.204176:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98a80 with x1875739641543680 00000100:00100000:1.0:1788845030.209095:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.209206:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641543808 00000100:00100000:0.0:1788845030.209216:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641543808 00002000:00100000:0.0:1788845030.209268:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482680 x1875739641543808/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.209280:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482680 with x1875739641543808 00000100:00100000:0.0:1788845030.209322:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 588 00000100:00100000:0.0:1788845030.209333:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641543808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.209802:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641543808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 470us (716us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.209810:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 588 00002000:00100000:0.0:1788845030.209818:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482680 x1875739641543808/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.209826:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482680 with x1875739641543808 00000100:00100000:1.0:1788845030.215816:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.215979:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641543936 00000100:00100000:2.0:1788845030.215990:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641543936 00002000:00100000:2.0:1788845030.216048:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480a80 x1875739641543936/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.216058:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480a80 with x1875739641543936 00000100:00100000:2.0:1788845030.216117:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 589 00000100:00100000:2.0:1788845030.216128:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641543936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.216630:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641543936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 502us (822us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.216640:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 589 00002000:00100000:2.0:1788845030.216647:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480a80 x1875739641543936/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.216655:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480a80 with x1875739641543936 00000100:00100000:2.0:1788845030.220548:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.220638:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641544064 00000100:00100000:3.0:1788845030.220648:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641544064 00002000:00100000:3.0:1788845030.220697:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410000 x1875739641544064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.220707:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410000 with x1875739641544064 00000100:00100000:3.0:1788845030.220791:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 590 00000100:00100000:3.0:1788845030.220802:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641544064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845030.221075:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641544064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (537us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845030.221082:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 590 00002000:00100000:3.0:1788845030.221089:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410000 x1875739641544064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.221113:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410000 with x1875739641544064 00000100:00100000:1.0:1788845030.225388:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.225516:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641544192 00000100:00100000:0.0:1788845030.225526:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641544192 00002000:00100000:0.0:1788845030.225574:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02483800 x1875739641544192/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.225584:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02483800 with x1875739641544192 00000100:00100000:0.0:1788845030.225622:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 591 00000100:00100000:0.0:1788845030.225631:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641544192:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.225854:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641544192:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 223us (474us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.225862:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 591 00002000:00100000:0.0:1788845030.225868:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02483800 x1875739641544192/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.225924:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02483800 with x1875739641544192 00000100:00100000:1.0:1788845030.230170:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.230457:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641544320 00000100:00100000:0.0:1788845030.230468:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641544320 00002000:00100000:0.0:1788845030.230517:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482d80 x1875739641544320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.230530:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482d80 with x1875739641544320 00000100:00100000:0.0:1788845030.230568:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 592 00000100:00100000:0.0:1788845030.230580:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641544320:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.230809:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641544320:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 230us (647us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.230817:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 592 00002000:00100000:0.0:1788845030.230823:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482d80 x1875739641544320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.230831:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482d80 with x1875739641544320 00000100:00100000:1.0:1788845030.234776:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.235048:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641544448 00000100:00100000:0.0:1788845030.235057:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641544448 00002000:00100000:0.0:1788845030.235109:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480000 x1875739641544448/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.235120:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480000 with x1875739641544448 00000100:00100000:0.0:1788845030.235160:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 593 00000100:00100000:0.0:1788845030.235171:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641544448:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.235577:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641544448:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 405us (815us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.235601:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 593 00002000:00100000:0.0:1788845030.235608:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480000 x1875739641544448/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.235616:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480000 with x1875739641544448 00000100:00100000:2.0:1788845030.240256:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.240362:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641544576 00000100:00100000:0.0:1788845030.240372:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641544576 00002000:00100000:0.0:1788845030.240537:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964380 x1875739641544576/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.240550:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964380 with x1875739641544576 00000100:00100000:0.0:1788845030.240628:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 594 00000100:00100000:0.0:1788845030.240639:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641544576:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845030.244594:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788845030.244603:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845030 -> 1788845030 00000004:00010000:0.0:1788845030.244605:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845030 -> 1788845026 00000004:00010000:0.0:1788845030.244607:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845030 -> 1788845030 00000004:00010000:0.0:1788845030.244610:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 520 00000100:00100000:0.0:1788845030.244684:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641544576:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4042us (4433us total) trans 4294967672 rc 0/0 00000100:00100000:0.0:1788845030.244696:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 594 00002000:00100000:0.0:1788845030.244708:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964380 x1875739641544576/t4294967672(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/448 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.244717:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964380 with x1875739641544576 00000100:00100000:2.0:1788845030.249272:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.249426:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641544704 00000100:00100000:3.0:1788845030.249437:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641544704 00000100:00100000:3.0:1788845030.249498:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 969 00000100:00100000:3.0:1788845030.249509:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641544704:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:2.0:1788845030.284112:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641544704:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 34594us (34841us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.284127:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 969 00000001:00080000:0.0:1788845030.284602:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967672 is committed 00000100:00100000:0.0:1788845030.287587:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.287714:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641544832 00000100:00100000:2.0:1788845030.287724:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641544832 00000100:00100000:2.0:1788845030.287791:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 970 00000100:00100000:2.0:1788845030.287802:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641544832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.287823:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.287844:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ace8f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86ea expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.287943:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.287987:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660255104:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.288068:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000004:00010000:0.0:1788845030.290705:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845026 -> 1788845030 00000004:00010000:0.0:1788845030.290714:0:11036:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 520 -> 584 00000100:00100000:0.0:1788845030.290740:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660255104:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.290764:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.290849:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ace8f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ea expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.290871:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.290874:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.290880:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ace8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ea expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.290930:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:0.0:1788845030.291029:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641544832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3228us (3449us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.291037:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 970 00000100:00100000:3.0:1788845030.294569:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.294674:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641544960 00000100:00100000:2.0:1788845030.294685:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641544960 00000100:00100000:2.0:1788845030.294774:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 971 00000100:00100000:2.0:1788845030.294850:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641544960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.294870:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.294963:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ace96 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86f1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.295006:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.295050:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660255232:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.295172:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.298619:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660255232:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.298645:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.298664:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ace96 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86f1 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.298687:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.298690:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.298695:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ace96 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86f1 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.298712:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:0.0:1788845030.298941:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641544960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4094us (4379us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.298950:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 971 00000100:00100000:0.0:1788845030.303954:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.304040:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641545088 00000100:00100000:2.0:1788845030.304050:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641545088 00000100:00100000:2.0:1788845030.304118:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 595 00000100:00100000:2.0:1788845030.304128:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641545088:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845030.304434:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788845030.304439:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845030 -> 1788845030 00000004:00010000:2.0:1788845030.304441:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845030 -> 1788845030 00000004:00010000:2.0:1788845030.304444:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845030 -> 1788845030 00000004:00010000:2.0:1788845030.304446:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:2.0:1788845030.304523:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641545088:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 396us (580us total) trans 4294967673 rc 0/0 00000100:00100000:2.0:1788845030.304532:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 595 00000100:00100000:3.0:1788845030.315459:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.315565:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641545216 00000100:00100000:0.0:1788845030.315576:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641545216 00000100:00100000:0.0:1788845030.315644:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 972 00000100:00100000:0.0:1788845030.315655:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641545216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.315675:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.315696:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ace9d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86f8 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.315734:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.315783:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660255360:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.315869:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.318155:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660255360:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.318248:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.318268:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ace9d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86f8 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.318290:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.318293:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.318299:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ace9d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86f8 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.318335:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:0.0:1788845030.318392:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641545216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2739us (2942us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.318401:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 972 00000100:00100000:1.0:1788845030.325380:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.325537:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641545344 00000100:00100000:3.0:1788845030.325546:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641545344 00000100:00100000:3.0:1788845030.325634:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 973 00000100:00100000:3.0:1788845030.325644:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641545344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845030.325664:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845030.325682:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287acea4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a86ff expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.325716:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845030.325755:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660255488:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845030.325834:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.330534:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660255488:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845030.330564:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845030.330584:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287acea4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ff expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.330607:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845030.330610:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845030.330614:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287acea4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ff expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.330629:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:2.0:1788845030.330735:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641545344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5091us (5364us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845030.330744:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 973 00000100:00100000:2.0:1788845030.343619:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.343745:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641545472 00000100:00100000:0.0:1788845030.343756:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641545472 00000100:00100000:0.0:1788845030.343825:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 974 00000100:00100000:0.0:1788845030.343836:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641545472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.343859:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.343930:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287aceab lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8706 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.343973:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.344014:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660255616:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.344095:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.352326:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660255616:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.352355:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.352374:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287aceab lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8706 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.352397:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.352400:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.352405:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287aceab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8706 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.352510:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:0.0:1788845030.352629:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641545472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8793us (9022us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.352638:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 974 00000100:00100000:3.0:1788845030.358199:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.358329:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641545600 00000100:00100000:0.0:1788845030.358342:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641545600 00000100:00100000:0.0:1788845030.358520:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 975 00000100:00100000:0.0:1788845030.358533:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641545600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.358556:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.358576:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287aceb2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a870d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.358613:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.358654:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660255744:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.358729:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.362219:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660255744:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.362247:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.362266:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287aceb2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a870d expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.362287:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.362290:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.362296:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287aceb2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a870d expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.362325:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:0.0:1788845030.362387:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641545600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3856us (4199us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.362395:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 975 00000100:00100000:1.0:1788845030.371636:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.371818:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641545728 00000100:00100000:2.0:1788845030.371829:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641545728 00000100:00100000:2.0:1788845030.371936:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 654 00000100:00100000:2.0:1788845030.371947:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641545728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.371966:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.371987:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287aceb9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8714 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.372048:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660255872:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.372124:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845030.392381:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660255872:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845030.392943:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287aceb9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8714 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.392997:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287aceb9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8714 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.393013:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845030.393094:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641545728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 21148us (21467us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845030.393102:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 654 00000100:00100000:0.0:1788845030.396182:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.396260:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641545856 00000100:00100000:3.0:1788845030.396271:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641545856 00002000:00100000:3.0:1788845030.396320:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f2680 x1875739641545856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.396330:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f2680 with x1875739641545856 00000100:00100000:3.0:1788845030.396381:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 596 00000100:00100000:3.0:1788845030.396392:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641545856:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845030.396656:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641545856:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (483us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845030.396665:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 596 00002000:00100000:3.0:1788845030.396672:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f2680 x1875739641545856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.396680:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f2680 with x1875739641545856 00000100:00100000:1.0:1788845030.405260:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.405448:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641545984 00000100:00100000:2.0:1788845030.405458:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641545984 00002000:00100000:2.0:1788845030.405524:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480380 x1875739641545984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.405536:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480380 with x1875739641545984 00000100:00100000:2.0:1788845030.405579:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 597 00000100:00100000:2.0:1788845030.405588:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641545984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.406168:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641545984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 578us (914us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.406176:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 597 00002000:00100000:2.0:1788845030.406183:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480380 x1875739641545984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.406191:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480380 with x1875739641545984 00000100:00100000:2.0:1788845030.412529:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.412693:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641546112 00000100:00100000:3.0:1788845030.412703:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641546112 00002000:00100000:3.0:1788845030.412754:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967480 x1875739641546112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.412787:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967480 with x1875739641546112 00000100:00100000:3.0:1788845030.412828:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 598 00000100:00100000:3.0:1788845030.412839:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641546112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845030.413132:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641546112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (612us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845030.413140:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 598 00002000:00100000:3.0:1788845030.413147:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967480 x1875739641546112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.413156:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967480 with x1875739641546112 00000100:00100000:2.0:1788845030.423252:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.423384:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641546240 00000100:00100000:0.0:1788845030.423396:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641546240 00002000:00100000:0.0:1788845030.423460:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966680 x1875739641546240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.423470:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966680 with x1875739641546240 00000100:00100000:0.0:1788845030.423509:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 599 00000100:00100000:0.0:1788845030.423518:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641546240:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.425050:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641546240:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 891us (1166us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.425062:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 599 00002000:00100000:0.0:1788845030.425072:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966680 x1875739641546240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.425082:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966680 with x1875739641546240 00000100:00100000:1.0:1788845030.433575:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.434353:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641546368 00000100:00100000:3.0:1788845030.434364:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641546368 00002000:00100000:2.0:1788845030.434911:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02483100 x1875739641546368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.434924:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02483100 with x1875739641546368 00000100:00100000:2.0:1788845030.435007:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 600 00000100:00100000:2.0:1788845030.435018:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641546368:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.435291:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641546368:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (1726us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.435298:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 600 00002000:00100000:2.0:1788845030.435305:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02483100 x1875739641546368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.435312:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02483100 with x1875739641546368 00000100:00100000:3.0:1788845030.440221:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.440938:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641546496 00000100:00100000:2.0:1788845030.440949:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641546496 00002000:00100000:2.0:1788845030.441000:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b100 x1875739641546496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.441011:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b100 with x1875739641546496 00000100:00100000:2.0:1788845030.441051:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 601 00000100:00100000:2.0:1788845030.441061:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641546496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.441312:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641546496:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 252us (1101us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.441320:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 601 00002000:00100000:2.0:1788845030.441328:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b100 x1875739641546496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.441336:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b100 with x1875739641546496 00000100:00100000:3.0:1788845030.452165:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.452281:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641546624 00000100:00100000:0.0:1788845030.452291:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641546624 00002000:00100000:0.0:1788845030.452340:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b800 x1875739641546624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.452351:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b800 with x1875739641546624 00000100:00100000:0.0:1788845030.452405:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 602 00000100:00100000:0.0:1788845030.452501:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641546624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.453189:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641546624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 687us (1027us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.453198:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 602 00002000:00100000:0.0:1788845030.453207:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b800 x1875739641546624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.453216:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b800 with x1875739641546624 00000100:00100000:0.0:1788845030.459842:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.460028:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641546752 00000100:00100000:2.0:1788845030.460039:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641546752 00002000:00100000:2.0:1788845030.460089:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f1c00 x1875739641546752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.460101:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1c00 with x1875739641546752 00000100:00100000:2.0:1788845030.460144:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 603 00000100:00100000:2.0:1788845030.460153:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641546752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.460429:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641546752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 275us (595us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.460438:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 603 00002000:00100000:2.0:1788845030.460445:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f1c00 x1875739641546752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.460474:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1c00 with x1875739641546752 00000100:00100000:3.0:1788845030.464803:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.464970:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641546880 00000100:00100000:1.0:1788845030.464981:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641546880 00002000:00100000:1.0:1788845030.465031:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f99f80 x1875739641546880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.465056:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f99f80 with x1875739641546880 00000100:00100000:1.0:1788845030.465096:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 604 00000100:00100000:1.0:1788845030.465106:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641546880:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.465385:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641546880:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (591us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.465394:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 604 00002000:00100000:1.0:1788845030.465402:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f99f80 x1875739641546880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.465411:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f99f80 with x1875739641546880 00000100:00100000:3.0:1788845030.475172:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.475276:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641547008 00000100:00100000:2.0:1788845030.475286:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641547008 00002000:00100000:2.0:1788845030.475336:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b480 x1875739641547008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.475347:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b480 with x1875739641547008 00000100:00100000:2.0:1788845030.475398:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 605 00000100:00100000:2.0:1788845030.475408:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641547008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.475681:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641547008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 273us (517us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.475689:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 605 00002000:00100000:2.0:1788845030.475695:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b480 x1875739641547008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.475701:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b480 with x1875739641547008 00000100:00100000:2.0:1788845030.480055:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.480158:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641547136 00000100:00100000:1.0:1788845030.480169:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641547136 00002000:00100000:1.0:1788845030.480224:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967b80 x1875739641547136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.480235:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967b80 with x1875739641547136 00000100:00100000:1.0:1788845030.480279:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 606 00000100:00100000:1.0:1788845030.480289:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641547136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.480667:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641547136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 378us (621us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.480676:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 606 00002000:00100000:1.0:1788845030.480684:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967b80 x1875739641547136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.480692:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967b80 with x1875739641547136 00000100:00100000:3.0:1788845030.486756:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.487170:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641547264 00000100:00100000:2.0:1788845030.487181:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641547264 00002000:00100000:2.0:1788845030.487231:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f98000 x1875739641547264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.487242:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98000 with x1875739641547264 00000100:00100000:2.0:1788845030.487285:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 607 00000100:00100000:2.0:1788845030.487294:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641547264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.487559:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641547264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 264us (813us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.487579:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 607 00002000:00100000:2.0:1788845030.487586:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f98000 x1875739641547264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.487593:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98000 with x1875739641547264 00000100:00100000:3.0:1788845030.493424:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.493575:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641547392 00000100:00100000:0.0:1788845030.493585:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641547392 00002000:00100000:0.0:1788845030.493636:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9a300 x1875739641547392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.493658:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9a300 with x1875739641547392 00000100:00100000:0.0:1788845030.493701:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 608 00000100:00100000:0.0:1788845030.493713:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641547392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.494067:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641547392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 355us (660us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.494076:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 608 00002000:00100000:0.0:1788845030.494083:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9a300 x1875739641547392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.494090:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9a300 with x1875739641547392 00000100:00100000:0.0:1788845030.498138:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.498232:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641547520 00000100:00100000:2.0:1788845030.498241:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641547520 00002000:00100000:2.0:1788845030.498288:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360efb80 x1875739641547520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.498300:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360efb80 with x1875739641547520 00000100:00100000:2.0:1788845030.498334:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 609 00000100:00100000:2.0:1788845030.498342:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641547520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.498587:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641547520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 245us (482us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.498595:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 609 00002000:00100000:2.0:1788845030.498602:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360efb80 x1875739641547520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.498610:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360efb80 with x1875739641547520 00000100:00100000:3.0:1788845030.502462:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.502610:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641547648 00000100:00100000:0.0:1788845030.502621:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641547648 00002000:00100000:0.0:1788845030.502682:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9bb80 x1875739641547648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.502709:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9bb80 with x1875739641547648 00000100:00100000:0.0:1788845030.502747:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 610 00000100:00100000:0.0:1788845030.502758:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641547648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.503071:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641547648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 313us (617us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.503079:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 610 00002000:00100000:0.0:1788845030.503086:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9bb80 x1875739641547648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.503095:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9bb80 with x1875739641547648 00000100:00100000:0.0:1788845030.507071:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.507183:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641547776 00000100:00100000:2.0:1788845030.507197:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641547776 00002000:00100000:2.0:1788845030.507254:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da49f80 x1875739641547776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.507266:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da49f80 with x1875739641547776 00000100:00100000:2.0:1788845030.507309:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 611 00000100:00100000:2.0:1788845030.507320:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641547776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.507973:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641547776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 652us (910us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.507983:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 611 00002000:00100000:2.0:1788845030.507991:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da49f80 x1875739641547776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.508000:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da49f80 with x1875739641547776 00000100:00100000:2.0:1788845030.512167:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.512273:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641547904 00000100:00100000:0.0:1788845030.512282:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641547904 00002000:00100000:0.0:1788845030.512332:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964700 x1875739641547904/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.512343:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964700 with x1875739641547904 00000100:00100000:0.0:1788845030.512386:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 612 00000100:00100000:0.0:1788845030.512395:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641547904:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.515601:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641547904:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 3193us (3430us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.515617:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 612 00002000:00100000:0.0:1788845030.515632:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964700 x1875739641547904/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.515642:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964700 with x1875739641547904 00000100:00100000:2.0:1788845030.526429:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.526777:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641548032 00000100:00100000:1.0:1788845030.526799:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641548032 00000100:00100000:1.0:1788845030.526865:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 976 00000100:00100000:1.0:1788845030.526876:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641548032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845030.526946:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845030.526970:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acec0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a871b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.527011:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845030.527055:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02483b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660256000:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845030.527146:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.535296:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02483b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660256000:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845030.535330:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845030.535351:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acec0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a871b expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.535374:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845030.535378:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845030.535384:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acec0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a871b expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.535400:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ac00, rc 0) 00000100:00100000:1.0:1788845030.535472:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641548032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8597us (9057us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845030.535481:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 976 00000100:00100000:2.0:1788845030.548857:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.549042:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641548160 00000100:00100000:3.0:1788845030.549054:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641548160 00000100:00100000:3.0:1788845030.549121:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 977 00000100:00100000:3.0:1788845030.549133:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641548160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845030.549156:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845030.549177:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287acec7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8722 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.549244:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845030.549291:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010482680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660256128:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845030.549387:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845030.559696:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010482680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660256128:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845030.559726:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845030.559745:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287acec7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8722 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.559766:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845030.559769:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845030.559774:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287acec7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8722 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.559789:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:3.0:1788845030.559870:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641548160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10740us (11030us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845030.559880:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 977 00000100:00100000:1.0:1788845030.567007:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.567371:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641548288 00000100:00100000:2.0:1788845030.567382:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641548288 00000100:00100000:2.0:1788845030.567453:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 655 00000100:00100000:2.0:1788845030.567464:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641548288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.567484:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.567501:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287acece lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8729 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.567563:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660256256:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.567658:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.573012:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660256256:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845030.573054:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287acece lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8729 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.573079:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287acece lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8729 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.573094:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:0.0:1788845030.573163:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641548288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5701us (6176us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.573172:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 655 00000100:00100000:3.0:1788845030.577997:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.578115:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641548416 00000100:00100000:2.0:1788845030.578126:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641548416 00000100:00100000:2.0:1788845030.578190:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 656 00000100:00100000:2.0:1788845030.578199:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641548416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.578218:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.578237:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287aced5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8730 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.578317:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660256384:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.578426:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.586025:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660256384:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845030.586070:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287aced5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8730 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.586097:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287aced5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8730 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.586113:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845030.586212:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641548416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8013us (8225us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845030.586222:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 656 00000100:00100000:0.0:1788845030.589703:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.590743:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641548544 00000100:00100000:2.0:1788845030.590756:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641548544 00000100:00100000:2.0:1788845030.590880:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 657 00000100:00100000:2.0:1788845030.590933:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641548544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.590954:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.590976:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acedc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8737 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.591038:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660256512:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.591116:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.594283:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660256512:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845030.594326:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acedc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8737 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.594352:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287acedc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8737 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.594378:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845030.594440:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641548544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3510us (4751us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845030.594448:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 657 00000100:00100000:1.0:1788845030.603656:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.603754:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641548672 00000100:00100000:2.0:1788845030.603776:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641548672 00002000:00100000:2.0:1788845030.603834:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480e00 x1875739641548672/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.603847:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480e00 with x1875739641548672 00000100:00100000:2.0:1788845030.603928:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 613 00000100:00100000:2.0:1788845030.603939:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641548672:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.604204:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641548672:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 267us (563us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.604213:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 613 00002000:00100000:2.0:1788845030.604220:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480e00 x1875739641548672/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.604229:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480e00 with x1875739641548672 00000100:00100000:3.0:1788845030.607688:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.611084:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641548800 00000100:00100000:2.0:1788845030.611119:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641548800 00002000:00100000:2.0:1788845030.611183:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480380 x1875739641548800/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.611195:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480380 with x1875739641548800 00000100:00100000:2.0:1788845030.611230:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 614 00000100:00100000:2.0:1788845030.611240:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641548800:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.611524:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641548800:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 285us (3846us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.611532:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 614 00002000:00100000:2.0:1788845030.611540:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480380 x1875739641548800/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.611548:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480380 with x1875739641548800 00000100:00100000:3.0:1788845030.614584:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660256640:0@lo:400:kworker.0 00000100:00100000:3.0:1788845030.617232:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845030.617437:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660256640 00000100:00100000:0.0:1788845030.617447:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660256640 00000100:00100000:0.0:1788845030.617506:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:0.0:1788845030.617516:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660256640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845030.621634:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660257024:0@lo:400:kworker.0 00000100:00100000:1.0:1788845030.621725:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845030.621794:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660257024 00000100:00100000:1.0:1788845030.621803:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660257024 00000100:00100000:1.0:1788845030.621872:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:1.0:1788845030.621909:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660257024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845030.621962:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660257024:12345-0@lo:400:kworker.0 Request processed in 81us (237us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845030.621969:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00080000:1.0:1788845030.621990:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845030.622020:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660257024:0@lo:400:kworker.0 00000100:00100000:1.0:1788845030.623917:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.623954:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641548928 00000100:00100000:1.0:1788845030.623961:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641548928 00002000:00100000:1.0:1788845030.624005:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481f80 x1875739641548928/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.624017:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481f80 with x1875739641548928 00000100:00100000:1.0:1788845030.624044:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 615 00000100:00100000:1.0:1788845030.624053:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641548928:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.624293:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641548928:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 240us (412us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.624301:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 615 00002000:00100000:1.0:1788845030.624308:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481f80 x1875739641548928/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.624336:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481f80 with x1875739641548928 00000100:00100000:0.0:1788845030.624548:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660256640:12345-0@lo:400:kworker.0 Request processed in 7032us (7323us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.624558:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:0.0:1788845030.624590:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660256768:0@lo:400:kworker.0 00000100:00100000:0.0:1788845030.624632:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845030.624677:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788845030.624685:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660256896:0@lo:400:kworker.0 00000100:00100000:0.0:1788845030.624708:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845030.624739:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660256768 00000100:00100000:0.0:1788845030.624749:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660256768 00000100:00100000:0.0:1788845030.624775:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660256896 00000100:00100000:0.0:1788845030.624778:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660256896 00000100:00100000:0.0:1788845030.624813:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1788845030.624820:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660256768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845030.624859:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660256768:12345-0@lo:400:kworker.0 Request processed in 39us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.624865:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1788845030.624879:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:0.0:1788845030.624911:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660256896:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845030.624942:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660256896:12345-0@lo:400:kworker.0 Request processed in 57us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.624948:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:0.0:1788845030.624985:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660256896:0@lo:400:kworker.0 00000100:00100000:0.0:1788845030.624997:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660256768:0@lo:400:kworker.0 00000100:00080000:0.0:1788845030.625058:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845030.625474:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788845030.625484:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660257280:0@lo:400:kworker.0 00000100:00100000:2.0:1788845030.625515:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845030.625545:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660257280 00000100:00100000:2.0:1788845030.625549:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660257280 00000100:00100000:2.0:1788845030.625585:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:2.0:1788845030.625592:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660257280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845030.625629:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660257280:12345-0@lo:400:kworker.0 Request processed in 39us (113us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.625636:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:1.0:1788845030.626129:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1788845030.626141:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660257152:0@lo:400:kworker.0 00000100:00100000:1.0:1788845030.626175:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845030.626206:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660257152 00000100:00100000:1.0:1788845030.626210:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660257152 00000100:00100000:1.0:1788845030.626244:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:1.0:1788845030.626250:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660257152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845030.626287:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660257152:12345-0@lo:400:kworker.0 Request processed in 37us (111us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845030.626293:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:1.0:1788845030.626320:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660257152:0@lo:400:kworker.0 00000100:00100000:2.0:1788845030.626442:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660257280:0@lo:400:kworker.0 00000100:00080000:2.0:1788845030.626458:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1788845030.626462:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845030.632066:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660256640:0@lo:400:kworker.0 00000100:00100000:1.0:1788845030.632392:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.632550:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641549056 00000100:00100000:2.0:1788845030.632563:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641549056 00002000:00100000:2.0:1788845030.632634:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849c00 x1875739641549056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.632966:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849c00 with x1875739641549056 00000100:00100000:2.0:1788845030.633007:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 616 00000100:00100000:2.0:1788845030.633018:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641549056:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.633471:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641549056:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 453us (1080us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.633479:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 616 00002000:00100000:2.0:1788845030.633488:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849c00 x1875739641549056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.633496:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849c00 with x1875739641549056 00000100:00100000:1.0:1788845030.637704:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.637926:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641549184 00000100:00100000:2.0:1788845030.637936:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641549184 00002000:00100000:2.0:1788845030.637983:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384a680 x1875739641549184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.637993:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384a680 with x1875739641549184 00000100:00100000:2.0:1788845030.638032:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 617 00000100:00100000:2.0:1788845030.638042:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641549184:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.638297:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641549184:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (603us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.638305:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 617 00002000:00100000:2.0:1788845030.638311:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384a680 x1875739641549184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.638339:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384a680 with x1875739641549184 00000100:00100000:0.0:1788845030.642400:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.642558:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641549312 00000100:00100000:1.0:1788845030.642568:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641549312 00000100:00100000:1.0:1788845030.642651:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 658 00000100:00100000:1.0:1788845030.642662:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641549312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845030.642680:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845030.642707:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287acee3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a873e expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845030.642784:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660257408:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845030.642865:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.645618:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660257408:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845030.645693:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287acee3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a873e expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.645719:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287acee3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a873e expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.645734:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ac00, rc 0) 00000100:00100000:1.0:1788845030.645812:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641549312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3151us (3421us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845030.645822:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 658 00000100:00100000:3.0:1788845030.649059:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.649413:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641549440 00000100:00100000:2.0:1788845030.649422:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641549440 00000100:00100000:2.0:1788845030.649485:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 659 00000100:00100000:2.0:1788845030.649495:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641549440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.649514:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.649533:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287aceea lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8745 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.649585:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660257536:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.649663:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.652235:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660257536:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845030.652291:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287aceea lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8745 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.652315:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287aceea lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8745 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.652329:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845030.657112:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641549440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7605us (8048us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845030.657125:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 659 00000100:00100000:3.0:1788845030.668688:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.668794:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641549568 00000100:00100000:1.0:1788845030.668805:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641549568 00002000:00100000:1.0:1788845030.668849:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482a00 x1875739641549568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 536/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.668860:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482a00 with x1875739641549568 00000100:00100000:1.0:1788845030.668932:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 618 00000100:00100000:1.0:1788845030.668943:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641549568:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.694191:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641549568:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 25238us (25502us total) trans 0 rc 860160/860160 00000100:00100000:1.0:1788845030.694205:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 618 00002000:00100000:1.0:1788845030.694221:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482a00 x1875739641549568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:136/0 lens 536/440 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 860160/860160 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.694231:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482a00 with x1875739641549568 00000100:00100000:2.0:1788845030.698489:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.698659:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641549696 00000100:00100000:0.0:1788845030.698669:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641549696 00000100:00100000:0.0:1788845030.698733:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 978 00000100:00100000:0.0:1788845030.698744:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641549696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.698867:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.698946:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acef1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a874c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.698986:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.699040:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660257664:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.699196:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.702552:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660257664:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845030.702601:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845030.702623:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acef1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a874c expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.702650:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845030.702653:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845030.702659:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acef1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a874c expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.702677:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:1.0:1788845030.702873:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641549696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4130us (4393us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845030.702926:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 978 00000100:00100000:2.0:1788845030.706022:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.706127:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641549824 00000100:00100000:1.0:1788845030.706136:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641549824 00000100:00100000:1.0:1788845030.706241:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 979 00000100:00100000:1.0:1788845030.706251:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641549824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845030.706347:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845030.706369:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acef8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8753 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.706485:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845030.706533:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660257792:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845030.706618:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.708868:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660257792:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845030.708956:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845030.708978:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acef8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8753 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.709000:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845030.709004:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845030.709008:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acef8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8753 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.709023:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ac00, rc 0) 00000100:00100000:1.0:1788845030.709072:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641549824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2822us (3064us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845030.709080:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 979 00000100:00100000:3.0:1788845030.712487:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.712597:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641549952 00000100:00100000:0.0:1788845030.712608:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641549952 00000100:00100000:0.0:1788845030.712669:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 980 00000100:00100000:0.0:1788845030.712681:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641549952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.712702:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.712722:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287aceff lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a875a expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.712759:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.712815:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660257920:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.712937:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.715961:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660257920:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.715990:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.716010:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287aceff lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a875a expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.716031:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.716035:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.716040:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287aceff lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a875a expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.716073:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030200, rc 0) 00000100:00100000:0.0:1788845030.716132:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641549952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3453us (3654us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.716140:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 980 00000100:00100000:3.0:1788845030.720228:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.720360:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641550080 00000100:00100000:0.0:1788845030.720371:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641550080 00000100:00100000:0.0:1788845030.720434:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 981 00000100:00100000:0.0:1788845030.720445:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641550080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.720465:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.720485:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acf06 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8761 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.720522:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.720563:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660258048:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.720647:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.723060:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660258048:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.723089:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.723108:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acf06 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8761 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.723128:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.723132:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.723138:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acf06 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8761 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.723165:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:0.0:1788845030.723269:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641550080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2826us (3049us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.723278:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 981 00000100:00100000:2.0:1788845030.728662:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.728746:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641550208 00000100:00100000:1.0:1788845030.728756:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641550208 00000100:00100000:1.0:1788845030.728828:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 982 00000100:00100000:1.0:1788845030.728839:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641550208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845030.728860:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845030.728920:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acf0d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8768 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.728960:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845030.729002:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660258176:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845030.729083:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.733829:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660258176:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845030.733873:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788845030.733934:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acf0d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8768 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.733957:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845030.733960:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845030.733966:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287acf0d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8768 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.733982:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ac00, rc 0) 00000100:00100000:1.0:1788845030.734042:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641550208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5204us (5391us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845030.734050:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 982 00000100:00100000:3.0:1788845030.736707:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.737161:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641550336 00000100:00100000:1.0:1788845030.737171:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641550336 00000100:00100000:1.0:1788845030.737224:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 983 00000100:00100000:1.0:1788845030.737234:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641550336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845030.737268:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845030.737290:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287acf14 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a876f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.737320:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.737331:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287acf14 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a876f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.737365:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.737398:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 15 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845030.737426:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660258304:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845030.737740:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.744229:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.744320:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641550464 00000100:00100000:0.0:1788845030.744332:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641550464 00010000:00010000:0.0:1788845030.744377:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 16 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.744429:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 354 00000100:00100000:0.0:1788845030.744441:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641550464:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845030.744456:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845030.744483:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 16 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.744501:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 16 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.744522:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.744528:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287acf14 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a876f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.744540:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287acf14 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a876f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.744569:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287acf14 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a876f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.744608:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287acf14 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a876f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.744621:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660258432:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845030.744695:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.747126:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660258432:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845030.747152:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845030.747271:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ace88 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86e3 expref: 16 pid: 5841 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845030.747290:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660258304:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845030.747310:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287acf14 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a876f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.747326:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845030.747329:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845030.747332:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1be00, rc 0) 00000100:00100000:0.0:1788845030.747429:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641550464:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2989us (3209us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.747440:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 354 00000100:00100000:1.0:1788845030.747594:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641550336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10361us (10895us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845030.747601:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 983 00000100:00100000:3.0:1788845030.750325:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.750477:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641550592 00000100:00100000:2.0:1788845030.750487:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641550592 00002000:00100000:2.0:1788845030.750539:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010483800 x1875739641550592/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.750551:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010483800 with x1875739641550592 00000100:00100000:2.0:1788845030.750627:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 619 00000100:00100000:2.0:1788845030.750638:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641550592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.751138:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641550592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 501us (822us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.751146:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 619 00002000:00100000:2.0:1788845030.751153:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010483800 x1875739641550592/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.751160:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010483800 with x1875739641550592 00000100:00100000:2.0:1788845030.754710:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.754871:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641550720 00000100:00100000:0.0:1788845030.754983:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641550720 00002000:00100000:0.0:1788845030.755037:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366300 x1875739641550720/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.755048:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366300 with x1875739641550720 00000100:00100000:0.0:1788845030.755108:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 620 00000100:00100000:0.0:1788845030.755118:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641550720:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.755388:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641550720:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 271us (685us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.755396:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 620 00002000:00100000:0.0:1788845030.755414:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366300 x1875739641550720/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.755437:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366300 with x1875739641550720 00000100:00100000:0.0:1788845030.758639:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.758719:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641550848 00000100:00100000:1.0:1788845030.758729:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641550848 00002000:00100000:1.0:1788845030.758786:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b8000 x1875739641550848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.758797:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b8000 with x1875739641550848 00000100:00100000:1.0:1788845030.758833:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 621 00000100:00100000:1.0:1788845030.758842:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641550848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.759301:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641550848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 457us (668us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.759308:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 621 00002000:00100000:1.0:1788845030.759315:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b8000 x1875739641550848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.759323:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b8000 with x1875739641550848 00000100:00100000:3.0:1788845030.762233:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.762582:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641550976 00000100:00100000:1.0:1788845030.762592:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641550976 00002000:00100000:1.0:1788845030.762642:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010481f80 x1875739641550976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.762654:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010481f80 with x1875739641550976 00000100:00100000:1.0:1788845030.762690:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 622 00000100:00100000:1.0:1788845030.762701:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641550976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.763000:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641550976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 301us (777us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.763009:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 622 00002000:00100000:1.0:1788845030.763016:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010481f80 x1875739641550976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.763023:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010481f80 with x1875739641550976 00000100:00100000:2.0:1788845030.767236:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.767284:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.767388:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641551104 00000100:00100000:0.0:1788845030.767398:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641551104 00000100:00100000:0.0:1788845030.767471:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 984 00000100:00100000:0.0:1788845030.767482:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641551104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.767503:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.767521:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8776 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.767555:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845030.767567:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.767575:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.767589:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.767591:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.767595:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030200, rc 0) 00000100:00100000:0.0:1788845030.767654:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641551104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 174us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.767661:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 984 00000100:00100000:1.0:1788845030.768571:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641551232 00000100:00100000:1.0:1788845030.768581:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641551232 00000100:00100000:1.0:1788845030.768630:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 660 00000100:00100000:1.0:1788845030.768639:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641551232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845030.768653:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845030.768680:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287acf22 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a877d expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845030.768736:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660258560:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845030.768820:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.771593:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660258560:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845030.771635:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287acf22 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a877d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.771661:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f29b7520/0xb2187301287acf22 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a877d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.771674:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ac00, rc 0) 00000100:00100000:1.0:1788845030.771783:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641551232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3144us (4497us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845030.771791:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 660 00000100:00100000:3.0:1788845030.774398:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.774528:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641551360 00000100:00100000:0.0:1788845030.774537:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641551360 00000100:00100000:0.0:1788845030.774593:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 661 00000100:00100000:0.0:1788845030.774603:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641551360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.774637:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.774656:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287acf29 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8784 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.774711:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359ba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660258688:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.774809:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845030.777049:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359ba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660258688:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845030.777088:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287acf29 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8784 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.777112:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287acf29 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8784 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845030.777126:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:1.0:1788845030.777184:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641551360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2582us (2793us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845030.777192:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 661 00000100:00100000:3.0:1788845030.780783:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.781223:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641551488 00000100:00100000:0.0:1788845030.781234:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641551488 00000100:00100000:0.0:1788845030.781489:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 985 00000100:00100000:0.0:1788845030.781501:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641551488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.781524:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.781528:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788845030.781574:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287acf14 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a876f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.781592:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.781597:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845030.781611:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a878b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.781637:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.781648:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.781683:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.781716:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 15 pid: 5839 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.781730:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660258816:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845030.781829:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845030.784612:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.784707:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660258816:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845030.784743:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.784774:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.784777:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.784782:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.784792:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:0.0:1788845030.784847:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641551488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3348us (4074us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845030.784855:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 985 00000100:00100000:2.0:1788845030.785078:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641551616 00000100:00100000:2.0:1788845030.785085:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641551616 00010000:00010000:2.0:1788845030.785116:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 16 pid: 5839 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.785152:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 355 00000100:00100000:2.0:1788845030.785160:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d4ac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641551616:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845030.785172:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845030.785209:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 16 pid: 5839 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.785227:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 16 pid: 5839 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.785248:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf1b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8776 expref: 16 pid: 5839 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.785270:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845030.785275:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.785285:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.785294:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.785326:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.785340:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660258944:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845030.785399:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845030.789605:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.789702:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660258944:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845030.789726:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845030.789789:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d4ac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641551616:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4629us (5186us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.789812:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 355 00000100:00100000:2.0:1788845030.789877:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641551744 00000100:00100000:2.0:1788845030.789914:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641551744 00002000:00100000:2.0:1788845030.789956:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480a80 x1875739641551744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.789967:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480a80 with x1875739641551744 00000100:00100000:2.0:1788845030.790001:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 623 00000100:00100000:2.0:1788845030.790010:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641551744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.790186:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641551744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 177us (590us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.790193:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 623 00002000:00100000:2.0:1788845030.790200:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480a80 x1875739641551744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.790208:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480a80 with x1875739641551744 00000100:00100000:3.0:1788845030.794821:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.795100:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641551872 00000100:00100000:0.0:1788845030.795110:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641551872 00000100:00100000:0.0:1788845030.795169:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 986 00000100:00100000:0.0:1788845030.795178:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641551872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.795198:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.795216:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf37 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8792 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.795267:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.795304:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660259072:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.795387:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.797389:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.797607:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660259072:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.797633:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.797654:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf37 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8792 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.797675:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.797678:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.797683:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf37 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8792 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.797697:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030e00, rc 0) 00000100:00100000:3.0:1788845030.797836:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641552000 00000100:00100000:3.0:1788845030.797844:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641552000 00000100:00100000:3.0:1788845030.797931:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 662 00000100:00100000:3.0:1788845030.797942:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641552000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845030.797957:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845030.797973:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287acf3e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8799 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845030.798019:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010480000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660259200:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.798204:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641551872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3026us (3392us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.798211:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 986 00000100:00100000:3.0:1788845030.798587:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788845030.801733:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1049696 -> 1079959 00000100:00100000:3.0:1788845030.801832:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010480000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660259200:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845030.801872:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287acf3e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8799 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.802183:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287acf3e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8799 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845030.802195:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:3.0:1788845030.802338:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641552000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4398us (4957us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845030.802347:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 662 00000100:00100000:1.0:1788845030.805642:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.805767:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.805958:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641552256 00000100:00100000:2.0:1788845030.805968:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641552256 00000100:00100000:2.0:1788845030.806023:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 663 00000100:00100000:2.0:1788845030.806033:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641552256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845030.806052:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845030.806069:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287acf45 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87a7 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845030.806124:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660259328:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845030.806214:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845030.807023:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641552128 00000100:00100000:0.0:1788845030.807030:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641552128 00000100:00100000:0.0:1788845030.807068:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 987 00000100:00100000:0.0:1788845030.807077:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641552128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845030.807089:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845030.807103:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf4c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87a0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.807145:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845030.807175:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660259456:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845030.807232:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845030.810458:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660259328:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845030.810497:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287acf45 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87a7 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.810522:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287acf45 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87a7 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845030.810536:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845030.810592:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641552256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4560us (4825us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845030.810600:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 663 00000100:00100000:0.0:1788845030.811326:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660259456:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845030.811343:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845030.811356:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf4c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87a0 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.811370:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845030.811373:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845030.811376:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf4c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87a0 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845030.811387:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030e00, rc 0) 00000100:00100000:0.0:1788845030.811438:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641552128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4363us (5804us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845030.811445:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 987 00000100:00100000:1.0:1788845030.816680:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.816794:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641552384 00000100:00100000:2.0:1788845030.816804:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641552384 00002000:00100000:2.0:1788845030.816858:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849f80 x1875739641552384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.816869:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849f80 with x1875739641552384 00000100:00100000:2.0:1788845030.816933:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 624 00000100:00100000:2.0:1788845030.816944:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641552384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.817388:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641552384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 443us (716us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.817396:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 624 00002000:00100000:2.0:1788845030.817404:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849f80 x1875739641552384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.817430:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849f80 with x1875739641552384 00000100:00100000:2.0:1788845030.825324:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.825473:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641552512 00000100:00100000:0.0:1788845030.825483:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641552512 00002000:00100000:0.0:1788845030.825533:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367480 x1875739641552512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.825545:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367480 with x1875739641552512 00000100:00100000:0.0:1788845030.825622:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 625 00000100:00100000:0.0:1788845030.825644:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641552512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.826053:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641552512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 409us (738us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.826062:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 625 00002000:00100000:0.0:1788845030.826069:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367480 x1875739641552512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.826077:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367480 with x1875739641552512 00000100:00100000:3.0:1788845030.829055:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.829126:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641552640 00000100:00100000:1.0:1788845030.829136:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641552640 00002000:00100000:1.0:1788845030.829181:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480000 x1875739641552640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.829192:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480000 with x1875739641552640 00000100:00100000:1.0:1788845030.829222:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 626 00000100:00100000:1.0:1788845030.829231:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641552640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.829623:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641552640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 391us (576us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.829631:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 626 00002000:00100000:1.0:1788845030.829638:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480000 x1875739641552640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.829646:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480000 with x1875739641552640 00000100:00100000:1.0:1788845030.833086:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845030.833421:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641552768 00000100:00100000:3.0:1788845030.833432:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641552768 00002000:00100000:3.0:1788845030.833478:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849180 x1875739641552768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.833489:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849180 with x1875739641552768 00000100:00100000:3.0:1788845030.833529:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 627 00000100:00100000:3.0:1788845030.833540:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641552768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845030.833752:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641552768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 213us (675us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845030.833772:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 627 00002000:00100000:3.0:1788845030.833779:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849180 x1875739641552768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845030.833786:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849180 with x1875739641552768 00000100:00100000:2.0:1788845030.838264:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.838375:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641552896 00000100:00100000:1.0:1788845030.838385:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641552896 00002000:00100000:1.0:1788845030.838506:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367800 x1875739641552896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.838516:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367800 with x1875739641552896 00000100:00100000:1.0:1788845030.838592:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 628 00000100:00100000:1.0:1788845030.838616:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641552896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.839057:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641552896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 440us (804us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.839065:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 628 00002000:00100000:1.0:1788845030.839074:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367800 x1875739641552896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.839085:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367800 with x1875739641552896 00000100:00100000:3.0:1788845030.842597:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.842754:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641553024 00000100:00100000:2.0:1788845030.842773:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641553024 00002000:00100000:2.0:1788845030.842826:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010483480 x1875739641553024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.842839:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010483480 with x1875739641553024 00000100:00100000:2.0:1788845030.842915:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 629 00000100:00100000:2.0:1788845030.842925:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641553024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.843366:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641553024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 441us (777us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.843374:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 629 00002000:00100000:2.0:1788845030.843382:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010483480 x1875739641553024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.843390:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010483480 with x1875739641553024 00000100:00100000:1.0:1788845030.847454:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.847649:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641553152 00000100:00100000:0.0:1788845030.847659:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641553152 00002000:00100000:0.0:1788845030.847710:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848e00 x1875739641553152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.847722:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848e00 with x1875739641553152 00000100:00100000:0.0:1788845030.847772:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 630 00000100:00100000:0.0:1788845030.847783:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641553152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.848032:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641553152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (587us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.848040:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 630 00002000:00100000:0.0:1788845030.848047:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848e00 x1875739641553152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.848055:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848e00 with x1875739641553152 00000100:00100000:1.0:1788845030.853696:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.853799:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641553280 00000100:00100000:2.0:1788845030.853809:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641553280 00002000:00100000:2.0:1788845030.853863:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848a80 x1875739641553280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.853875:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848a80 with x1875739641553280 00000100:00100000:2.0:1788845030.854105:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 631 00000100:00100000:2.0:1788845030.854115:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641553280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.854537:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641553280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 421us (848us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.854558:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 631 00002000:00100000:2.0:1788845030.854566:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848a80 x1875739641553280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.854573:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848a80 with x1875739641553280 00000100:00100000:2.0:1788845030.859838:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.859983:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641553408 00000100:00100000:0.0:1788845030.860057:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641553408 00002000:00100000:0.0:1788845030.860112:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02365f80 x1875739641553408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.860146:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02365f80 with x1875739641553408 00000100:00100000:0.0:1788845030.860223:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 632 00000100:00100000:0.0:1788845030.860232:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641553408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.863058:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641553408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 2817us (3222us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.863071:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 632 00002000:00100000:0.0:1788845030.863083:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02365f80 x1875739641553408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.863094:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02365f80 with x1875739641553408 00000100:00100000:3.0:1788845030.867340:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.867525:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641553536 00000100:00100000:1.0:1788845030.867538:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641553536 00002000:00100000:1.0:1788845030.867594:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010481880 x1875739641553536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.867607:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010481880 with x1875739641553536 00000100:00100000:1.0:1788845030.867641:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 633 00000100:00100000:1.0:1788845030.867652:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641553536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.867876:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641553536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 224us (547us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.867916:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 633 00002000:00100000:1.0:1788845030.867924:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010481880 x1875739641553536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.867932:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010481880 with x1875739641553536 00000100:00100000:3.0:1788845030.871395:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.871506:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641553664 00000100:00100000:1.0:1788845030.871516:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641553664 00002000:00100000:1.0:1788845030.871563:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480e00 x1875739641553664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.871583:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480e00 with x1875739641553664 00000100:00100000:1.0:1788845030.871620:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 634 00000100:00100000:1.0:1788845030.871628:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641553664:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.871843:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641553664:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 215us (459us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.871851:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 634 00002000:00100000:1.0:1788845030.871858:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480e00 x1875739641553664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.871866:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480e00 with x1875739641553664 00000100:00100000:2.0:1788845030.876092:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.876197:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641553792 00000100:00100000:1.0:1788845030.876207:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641553792 00002000:00100000:1.0:1788845030.876257:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367100 x1875739641553792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.876268:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367100 with x1875739641553792 00000100:00100000:1.0:1788845030.876311:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 635 00000100:00100000:1.0:1788845030.876321:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641553792:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.876547:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641553792:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 226us (463us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.876555:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 635 00002000:00100000:1.0:1788845030.876561:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367100 x1875739641553792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.876569:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367100 with x1875739641553792 00000100:00100000:3.0:1788845030.880307:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.880423:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641553920 00000100:00100000:1.0:1788845030.880445:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641553920 00002000:00100000:1.0:1788845030.880498:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482d80 x1875739641553920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.880509:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482d80 with x1875739641553920 00000100:00100000:1.0:1788845030.880547:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 636 00000100:00100000:1.0:1788845030.880556:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641553920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.880769:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641553920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 205us (462us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.880776:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 636 00002000:00100000:1.0:1788845030.880784:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482d80 x1875739641553920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.880792:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482d80 with x1875739641553920 00000100:00100000:3.0:1788845030.884524:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.884857:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641554048 00000100:00100000:1.0:1788845030.884867:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641554048 00002000:00100000:1.0:1788845030.884965:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482680 x1875739641554048/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 504/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.884978:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482680 with x1875739641554048 00000100:00100000:1.0:1788845030.885006:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 637 00000100:00100000:1.0:1788845030.885015:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554048:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845030.889715:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788845030.889725:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845030 -> 1788845030 00000004:00010000:1.0:1788845030.889727:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845030 -> 1788845030 00000004:00010000:1.0:1788845030.889730:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845030 -> 1788845030 00000004:00010000:1.0:1788845030.889732:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:1.0:1788845030.889824:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641554048:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4804us (5304us total) trans 4294967674 rc 0/0 00000100:00100000:1.0:1788845030.889845:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 637 00002000:00100000:1.0:1788845030.889859:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482680 x1875739641554048/t4294967674(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 504/448 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.889870:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482680 with x1875739641554048 00000100:00100000:3.0:1788845030.893295:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.893391:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641554176 00000100:00100000:2.0:1788845030.893408:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641554176 00000100:00100000:2.0:1788845030.893525:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 988 00000100:00100000:2.0:1788845030.893536:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641554176:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:0.0:1788845030.916548:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967674 is committed 00000001:00080000:0.0:1788845030.916568:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967673 is committed 00000100:00100000:2.0:1788845030.916645:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554176:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 23105us (23354us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845030.916659:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 988 00000100:00100000:3.0:1788845030.919701:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.919795:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641554304 00000100:00100000:1.0:1788845030.919805:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641554304 00002000:00100000:1.0:1788845030.919850:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036081180 x1875739641554304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.919858:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036081180 with x1875739641554304 00000100:00100000:3.0:1788845030.919921:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 638 00000100:00100000:3.0:1788845030.919933:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554304:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.920303:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554304:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 373us (609us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.920309:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 638 00002000:00100000:2.0:1788845030.920315:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036081180 x1875739641554304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.920320:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036081180 with x1875739641554304 00000100:00100000:0.0:1788845030.922806:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.923146:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641554432 00000100:00100000:2.0:1788845030.923156:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641554432 00002000:00100000:2.0:1788845030.923205:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359baa00 x1875739641554432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.923217:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359baa00 with x1875739641554432 00000100:00100000:2.0:1788845030.923264:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 639 00000100:00100000:2.0:1788845030.923274:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.924133:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 854us (1331us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.924142:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 639 00002000:00100000:2.0:1788845030.924151:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359baa00 x1875739641554432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.924160:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359baa00 with x1875739641554432 00000100:00100000:3.0:1788845030.928206:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.928379:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641554560 00000100:00100000:1.0:1788845030.928388:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641554560 00002000:00100000:1.0:1788845030.928445:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082300 x1875739641554560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.928456:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082300 with x1875739641554560 00000100:00100000:1.0:1788845030.928494:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 640 00000100:00100000:1.0:1788845030.928504:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.928723:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 220us (527us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.928730:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 640 00002000:00100000:1.0:1788845030.928737:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082300 x1875739641554560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.928744:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082300 with x1875739641554560 00000100:00100000:3.0:1788845030.932055:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.932190:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641554688 00000100:00100000:1.0:1788845030.932200:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641554688 00002000:00100000:1.0:1788845030.932264:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036083480 x1875739641554688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.932274:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036083480 with x1875739641554688 00000100:00100000:1.0:1788845030.932304:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 641 00000100:00100000:1.0:1788845030.932314:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641554688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.932802:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641554688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 488us (809us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.932811:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 641 00002000:00100000:1.0:1788845030.932818:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036083480 x1875739641554688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.932826:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036083480 with x1875739641554688 00000100:00100000:0.0:1788845030.937645:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.938038:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641554816 00000100:00100000:1.0:1788845030.938049:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641554816 00002000:00100000:1.0:1788845030.938121:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9500 x1875739641554816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.938133:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9500 with x1875739641554816 00000100:00100000:1.0:1788845030.938173:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 642 00000100:00100000:1.0:1788845030.938185:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.938500:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641554816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 316us (864us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.938509:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 642 00002000:00100000:1.0:1788845030.938516:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9500 x1875739641554816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.938524:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9500 with x1875739641554816 00000100:00100000:3.0:1788845030.943700:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845030.943766:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641554944 00000100:00100000:2.0:1788845030.943776:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641554944 00002000:00100000:2.0:1788845030.943821:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036081880 x1875739641554944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.943829:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036081880 with x1875739641554944 00000100:00100000:2.0:1788845030.943864:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 643 00000100:00100000:2.0:1788845030.943874:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641554944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845030.944286:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641554944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 414us (593us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845030.944292:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 643 00002000:00100000:2.0:1788845030.944298:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036081880 x1875739641554944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845030.944303:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036081880 with x1875739641554944 00000100:00100000:2.0:1788845030.952550:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.952848:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641555072 00000100:00100000:0.0:1788845030.952861:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641555072 00002000:00100000:0.0:1788845030.952960:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367b80 x1875739641555072/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.952969:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367b80 with x1875739641555072 00000100:00100000:0.0:1788845030.952999:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 644 00000100:00100000:0.0:1788845030.953007:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555072:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.953245:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555072:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 237us (705us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.953252:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 644 00002000:00100000:0.0:1788845030.953259:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367b80 x1875739641555072/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.953282:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367b80 with x1875739641555072 00000100:00100000:0.0:1788845030.958865:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.959020:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641555200 00000100:00100000:1.0:1788845030.959031:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641555200 00002000:00100000:1.0:1788845030.959083:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9c00 x1875739641555200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.959094:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9c00 with x1875739641555200 00000100:00100000:1.0:1788845030.959123:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 645 00000100:00100000:1.0:1788845030.959133:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.959407:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (553us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.959560:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 645 00002000:00100000:1.0:1788845030.959569:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9c00 x1875739641555200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.959577:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9c00 with x1875739641555200 00000100:00100000:3.0:1788845030.964332:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.964623:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641555328 00000100:00100000:0.0:1788845030.964633:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641555328 00002000:00100000:0.0:1788845030.964681:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036080a80 x1875739641555328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.964693:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036080a80 with x1875739641555328 00000100:00100000:0.0:1788845030.964735:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 646 00000100:00100000:0.0:1788845030.964745:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641555328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.965363:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641555328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 617us (1040us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.965371:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 646 00002000:00100000:0.0:1788845030.965379:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036080a80 x1875739641555328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.965386:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036080a80 with x1875739641555328 00000100:00100000:3.0:1788845030.977664:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.977840:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641555456 00000100:00100000:1.0:1788845030.977851:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641555456 00002000:00100000:1.0:1788845030.977956:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082a00 x1875739641555456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.977966:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082a00 with x1875739641555456 00000100:00100000:1.0:1788845030.978006:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 647 00000100:00100000:1.0:1788845030.978015:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.978279:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 264us (623us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.978287:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 647 00002000:00100000:1.0:1788845030.978295:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082a00 x1875739641555456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.978320:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082a00 with x1875739641555456 00000100:00100000:2.0:1788845030.982873:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845030.983220:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641555584 00000100:00100000:0.0:1788845030.983230:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641555584 00002000:00100000:0.0:1788845030.983278:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364700 x1875739641555584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.983289:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364700 with x1875739641555584 00000100:00100000:0.0:1788845030.983332:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 648 00000100:00100000:0.0:1788845030.983342:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845030.983805:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 463us (940us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845030.983827:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 648 00002000:00100000:0.0:1788845030.983835:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364700 x1875739641555584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845030.983843:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364700 with x1875739641555584 00000100:00100000:3.0:1788845030.990354:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845030.991157:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641555712 00000100:00100000:1.0:1788845030.991168:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641555712 00002000:00100000:1.0:1788845030.991224:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036083800 x1875739641555712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/0 e 0 to 0 dl 1788845041 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.991235:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036083800 with x1875739641555712 00000100:00100000:1.0:1788845030.991278:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 649 00000100:00100000:1.0:1788845030.991288:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845030.991565:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 278us (1221us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845030.991573:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 649 00002000:00100000:1.0:1788845030.991580:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036083800 x1875739641555712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:136/0 lens 488/4536 e 0 to 0 dl 1788845041 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845030.991587:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036083800 with x1875739641555712 00010000:00010000:1.0:1788845030.994710:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845031.003373:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.003730:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641555840 00000100:00100000:0.0:1788845031.003740:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641555840 00002000:00100000:0.0:1788845031.003808:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366a00 x1875739641555840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.003819:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366a00 with x1875739641555840 00000100:00100000:0.0:1788845031.003863:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 650 00000100:00100000:0.0:1788845031.003875:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555840:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845031.004209:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555840:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 336us (847us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845031.004216:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 650 00002000:00100000:0.0:1788845031.004223:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366a00 x1875739641555840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.004230:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366a00 with x1875739641555840 00000100:00080000:2.0:1788845031.006448:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845031.009296:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.010421:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641555968 00000100:00100000:2.0:1788845031.010435:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641555968 00002000:00100000:2.0:1788845031.010495:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036081c00 x1875739641555968/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.010507:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036081c00 with x1875739641555968 00000100:00100000:2.0:1788845031.010560:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 651 00000100:00100000:2.0:1788845031.010570:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555968:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845031.010860:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641555968:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 290us (1573us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845031.010868:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 651 00002000:00100000:2.0:1788845031.010875:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036081c00 x1875739641555968/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.010919:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036081c00 with x1875739641555968 00000100:00100000:3.0:1788845031.020078:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.020239:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641556096 00000100:00100000:0.0:1788845031.020252:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641556096 00002000:00100000:0.0:1788845031.020306:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036080000 x1875739641556096/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.020319:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036080000 with x1875739641556096 00000100:00100000:0.0:1788845031.020365:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 652 00000100:00100000:0.0:1788845031.020376:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641556096:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845031.025559:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788845031.025568:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845030 -> 1788845031 00000004:00010000:0.0:1788845031.025571:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845030 -> 1788845030 00000004:00010000:0.0:1788845031.025574:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845030 -> 1788845031 00000004:00010000:0.0:1788845031.025576:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:0.0:1788845031.025646:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641556096:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5267us (5573us total) trans 4294967675 rc 0/0 00000100:00100000:0.0:1788845031.025657:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 652 00002000:00100000:0.0:1788845031.025671:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036080000 x1875739641556096/t4294967675(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:137/0 lens 488/448 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.025682:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036080000 with x1875739641556096 00000100:00100000:3.0:1788845031.032985:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.033530:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641556224 00000100:00100000:1.0:1788845031.033542:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641556224 00000100:00100000:1.0:1788845031.033594:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 989 00000100:00100000:1.0:1788845031.033603:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641556224:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:0.0:1788845031.084433:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967675 is committed 00000100:00100000:2.0:1788845031.084681:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641556224:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 51072us (51699us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845031.084694:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 989 00000100:00100000:3.0:1788845031.094039:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.097185:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.097290:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641556352 00000100:00100000:0.0:1788845031.097300:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641556352 00000100:00100000:0.0:1788845031.097384:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 990 00000100:00100000:0.0:1788845031.097396:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641556352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.097429:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.097450:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf53 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87ae expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.097488:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.097532:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660259584:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.097618:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00080000:1.0:1788845031.098214:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845031.100297:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641556480 00000100:00100000:2.0:1788845031.100313:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641556480 00000100:00100000:2.0:1788845031.100406:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 664 00000100:00100000:2.0:1788845031.100418:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641556480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.100437:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.100457:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acf5a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87b5 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.100519:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660259712:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.100618:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00080000:1.0:1788845031.100989:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1788845031.117964:0:6685:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845030 -> 1788845031 00002000:00010000:2.0:1788845031.117973:0:6685:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845030 -> 1788845031 00000100:00100000:2.0:1788845031.117999:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660259712:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.118037:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acf5a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87b5 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.118062:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acf5a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87b5 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.118079:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.118161:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641556480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17746us (24132us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.118170:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 664 00000004:00010000:0.0:1788845031.118319:0:5839:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 584 -> 712 00000100:00100000:0.0:1788845031.118337:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660259584:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845031.118352:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845031.118364:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf53 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ae expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.118378:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.118381:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845031.118386:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030200/0xb2187301287acf53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ae expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.118397:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030200, rc 0) 00000100:00100000:0.0:1788845031.118423:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641556352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 21029us (21247us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845031.118430:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 990 00000100:00100000:0.0:1788845031.121867:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.121971:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.122231:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641556736 00000100:00100000:1.0:1788845031.122241:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641556736 00000100:00100000:1.0:1788845031.122302:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 665 00000100:00100000:3.0:1788845031.122306:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641556608 00000100:00100000:3.0:1788845031.122312:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641556608 00000100:00100000:1.0:1788845031.122316:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641556736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845031.122350:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1788845031.122354:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 991 00000100:00100000:3.0:1788845031.122364:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641556608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845031.122371:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287acf61 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87c3 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.122376:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.122392:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287acf68 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87bc expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845031.122426:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03848000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660259968:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845031.122441:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.122477:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036083b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660260096:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845031.122511:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.122512:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.131230:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660259840:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845031.131309:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845031.131558:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660259840 00000100:00100000:3.0:1788845031.131567:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660259840 00000100:00100000:3.0:1788845031.131634:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 79 00000100:00100000:3.0:1788845031.131645:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660259840:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788845031.131663:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03848000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660259968:192.168.201.29@tcp:106: 00002000:00000024:3.0:1788845031.131671:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 108986368 pending 0 free 3859808256 avail 3850371072 00010000:00010000:2.0:1788845031.131703:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287acf61 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87c3 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.131732:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287acf61 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87c3 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.131742:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660259840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 97us (429us total) trans 0 rc 0/0 00010000:00010000:2.0:1788845031.131751:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ba00, rc 0) 00000100:00100000:2.0:1788845031.131796:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641556736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9483us (9938us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.131826:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 665 00000100:00100000:0.0:1788845031.131963:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660259840:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845031.132011:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036083b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660260096:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845031.132025:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845031.132038:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287acf68 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87bc expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.132052:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.132062:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845031.132067:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287acf68 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87bc expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.132078:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:0.0:1788845031.132101:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641556608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9740us (10131us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845031.132109:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 991 00000100:00100000:3.0:1788845031.135685:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 79 00000100:00100000:1.0:1788845031.144204:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.144441:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641556992 00000100:00100000:3.0:1788845031.144453:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641556992 00000100:00100000:3.0:1788845031.144518:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 992 00000100:00100000:3.0:1788845031.144529:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641556992:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845031.144598:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845031.144625:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287ace57 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86b2 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.144645:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.144650:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845031.145740:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.145766:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.145798:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.145835:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.145856:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 10 pid: 5839 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.145870:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660260224:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845031.145981:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.149621:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660260224:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845031.149654:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845031.151727:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.153142:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641557120 00000100:00100000:2.0:1788845031.153154:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641557120 00010000:00010000:2.0:1788845031.153200:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 11 pid: 5839 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.153250:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 356 00000100:00100000:2.0:1788845031.153261:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641557120:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845031.153276:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845031.153284:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 11 pid: 5839 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.153301:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 11 pid: 5839 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.153323:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ace49 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86ab expref: 11 pid: 5839 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.153339:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.153345:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.153355:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.153362:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.153415:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845031.153482:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641557120:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 221us (1763us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845031.153489:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 356 00010000:00010000:3.0:1788845031.153631:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.153639:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845031.153767:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967676 00000004:00010000:3.0:1788845031.153942:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845031.153946:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845031.153949:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845030 -> 1788845030 00000004:00010000:3.0:1788845031.153951:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845031.153953:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:3.0:1788845031.153967:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bab12921 reply state 00000000d8407bf0 transno 4294967677 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.153977:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845031.154057:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641556992:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 9530us (9865us total) trans 4294967677 rc 0/0 00000100:00100000:3.0:1788845031.154064:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 992 00000020:00080000:3.0:1788845031.154067:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845031.157084:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.157104:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.157113:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.157119:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d15c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845031.157124:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.157131:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845031.157149:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:2.0:1788845031.157150:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.157155:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845031.157160:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.157166:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287acf6f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.157239:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641557248 00000100:00100000:3.0:1788845031.157249:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641557248 00000100:00100000:3.0:1788845031.157479:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 993 00000100:00100000:3.0:1788845031.157492:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641557248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.157516:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.157533:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87ca expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.157561:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.157571:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.157605:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.157622:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 9 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.157636:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660260352:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845031.157749:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.162071:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.162172:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641557376 00000100:00100000:2.0:1788845031.162181:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641557376 00010000:00010000:2.0:1788845031.162234:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.162285:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 357 00000100:00100000:2.0:1788845031.162295:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641557376:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845031.162313:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845031.162339:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.162358:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.162382:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.162389:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.162401:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.162410:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.162448:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.162462:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660260480:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845031.162530:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00010000:00010000:3.0:1788845031.163364:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acf30 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a878b expref: 10 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.163383:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660260352:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845031.163407:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.163419:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.163423:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845031.163426:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:3.0:1788845031.163478:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641557248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5989us (6333us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845031.163486:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 993 00000100:00100000:2.0:1788845031.189793:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660260480:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845031.189823:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845031.189873:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641557376:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 27579us (27810us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845031.189921:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 357 00000100:00100000:0.0:1788845031.194615:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.194704:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641557504 00000100:00100000:3.0:1788845031.194715:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641557504 00000100:00100000:3.0:1788845031.194789:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 666 00000100:00100000:3.0:1788845031.194800:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641557504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.194824:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.194844:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87d1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.194927:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.194938:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(11 5) 00010000:00010000:3.0:1788845031.194957:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:3.0:1788845031.194964:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.194999:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.195017:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 5 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.195032:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660260608:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845031.195113:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.214164:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036080e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660260608:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845031.214204:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.214226:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:3.0:1788845031.214283:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641557504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 19484us (19677us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845031.214292:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 666 00000100:00100000:1.0:1788845031.214607:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.214743:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641557632 00000100:00100000:2.0:1788845031.214756:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641557632 00002000:00100000:2.0:1788845031.214819:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1079958) req@ffff949f03412d80 x1875739641557632/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.214828:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03412d80 with x1875739641557632 ext(1048576->1079958) 00010000:00010000:2.0:1788845031.214836:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 6 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.214855:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 6 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845031.214862:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f03412d80 00000100:00100000:2.0:1788845031.214931:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 18 00000100:00100000:2.0:1788845031.214941:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641557632:12345-192.168.201.29@tcp:4:fsx.0 00000100:00080000:1.0:1788845031.215162:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845031.218041:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641557632:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3093us (3426us total) trans 4294967420 rc 0/0 00000100:00100000:2.0:1788845031.218054:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 18 00002000:00100000:2.0:1788845031.218065:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1079958) req@ffff949f03412d80 x1875739641557632/t4294967420(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:137/0 lens 488/448 e 0 to 0 dl 1788845042 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.218075:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03412d80 with x1875739641557632 ext(1048576->1079958) 00010000:00010000:2.0:1788845031.218096:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 8 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.218117:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 8 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845031.218126:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f03412d80 00000100:00100000:1.0:1788845031.227991:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.228177:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641557760 00000100:00100000:2.0:1788845031.228188:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641557760 00010000:00010000:2.0:1788845031.228231:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 8 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.228278:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 358 00000100:00100000:2.0:1788845031.228290:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641557760:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845031.228306:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845031.228329:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 8 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.228348:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 8 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.228370:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ace3b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a86a4 expref: 8 pid: 6684 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.228386:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:2.0:1788845031.228390:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:2.0:1788845031.228414:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.228424:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.228463:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.228477:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660260736:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845031.228569:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.234091:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.234246:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660260736:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845031.234283:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845031.234340:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641557760:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6052us (6358us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845031.234351:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 358 00000100:00100000:0.0:1788845031.234749:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641557888 00000100:00100000:0.0:1788845031.234758:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641557888 00002000:00010000:0.0:1788845031.234793:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1337909->18446744073709551615) 00002000:00100000:0.0:1788845031.234804:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036080e00 with x1875739641557888 ext(1337909->18446744073709551615) 00002000:00010000:0.0:1788845031.234815:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036080e00. 00000100:00100000:0.0:1788845031.234848:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 371 00000100:00100000:0.0:1788845031.234858:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641557888:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845031.235134:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1079959 -> 1337909 00002000:00010000:0.0:1788845031.235139:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845031 -> 1788845031 00002000:00010000:0.0:1788845031.235141:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845030 -> 1788845029 00002000:00010000:0.0:1788845031.235143:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845031 -> 1788845031 00002000:00010000:0.0:1788845031.235145:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 129 -> 15 00000100:00100000:0.0:1788845031.235179:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641557888:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 323us (1097us total) trans 4294967421 rc 0/0 00000100:00100000:0.0:1788845031.235186:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 371 00002000:00010000:0.0:1788845031.235190:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1337909->18446744073709551615) 00002000:00100000:0.0:1788845031.235207:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036080e00 with x1875739641557888 ext(1337909->18446744073709551615) 00002000:00010000:0.0:1788845031.235212:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036080e00. 00000100:00100000:3.0:1788845031.243816:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.243928:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641558016 00000100:00100000:0.0:1788845031.243938:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641558016 00000100:00100000:0.0:1788845031.244006:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 994 00000100:00100000:0.0:1788845031.244018:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641558016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.244043:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.244064:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acf84 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87d8 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.244203:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf8b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.244220:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf8b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.244230:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf8b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845031.244256:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1337909 00000004:00200000:0.0:1788845031.244260:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845031.244277:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf8b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.244303:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acf84 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d8 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.244322:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf8b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d8 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.244349:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.244352:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845031.244355:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030e00, rc 0) 00000100:00100000:0.0:1788845031.244421:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641558016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 404us (616us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845031.244428:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 994 00000100:00100000:1.0:1788845031.262511:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.262617:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641558144 00000100:00100000:2.0:1788845031.262627:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641558144 00002000:00100000:2.0:1788845031.262686:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f2d80 x1875739641558144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.262697:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f2d80 with x1875739641558144 00000100:00100000:2.0:1788845031.262734:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 653 00000100:00100000:2.0:1788845031.262745:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641558144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845031.267456:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.268925:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641558272 00000100:00100000:2.0:1788845031.268939:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641558272 00002000:00100000:2.0:1788845031.269005:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1339391) req@ffff94a036081f80 x1875739641558272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.269017:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036081f80 with x1875739641558272 ext(1048576->1339391) 00002000:00010000:2.0:1788845031.269022:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036081f80 00000100:00100000:2.0:1788845031.269059:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 372 00000100:00100000:2.0:1788845031.269069:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641558272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845031.290079:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641558144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 27323us (27566us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1788845031.290093:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 653 00002000:00100000:2.0:1788845031.290107:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f2d80 x1875739641558144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/440 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.290117:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f2d80 with x1875739641558144 00000100:00100000:2.0:1788845031.297155:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641558272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 28075us (29697us total) trans 0 rc 290816/290816 00000100:00100000:2.0:1788845031.297172:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 372 00002000:00100000:2.0:1788845031.297186:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1339391) req@ffff94a036081f80 x1875739641558272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/440 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 290816/290816 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.297197:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036081f80 with x1875739641558272 ext(1048576->1339391) 00002000:00010000:2.0:1788845031.297204:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036081f80 00000100:00100000:1.0:1788845031.303789:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.303970:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641558400 00000100:00100000:0.0:1788845031.303981:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641558400 00000100:00100000:0.0:1788845031.304048:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 995 00000100:00100000:0.0:1788845031.304059:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641558400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.304082:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.304104:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acf92 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87df expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.304198:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.304215:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.304228:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845031.304257:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1337909 00000004:00200000:0.0:1788845031.304262:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845031.304284:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.304327:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acf92 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.304347:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.304357:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.304361:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845031.304365:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031200, rc 0) 00000100:00100000:0.0:1788845031.304448:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641558400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 391us (668us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845031.304455:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 995 00000100:00100000:1.0:1788845031.317357:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.317414:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.317866:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641558656 00000100:00100000:2.0:1788845031.317904:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641558656 00000100:00100000:2.0:1788845031.317971:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 667 00000100:00100000:2.0:1788845031.317981:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641558656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.318001:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.318035:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfa0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87ed expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.318096:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660260864:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.318190:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.318679:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641558528 00000100:00100000:0.0:1788845031.318685:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641558528 00000100:00100000:0.0:1788845031.318725:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 996 00000100:00100000:0.0:1788845031.318733:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641558528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.318746:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.318773:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acfa7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87e6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.318804:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.318831:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660260992:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.318915:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00002000:00010000:2.0:1788845031.327333:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845031 00000100:00100000:2.0:1788845031.327381:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660260864:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.327423:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfa0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ed expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.327449:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfa0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ed expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.327465:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.327535:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641558656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9555us (10120us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.327545:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 667 00000004:00010000:3.0:1788845031.328178:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845030 -> 1788845031 00000100:00100000:3.0:1788845031.328199:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660260992:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845031.328249:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845031.328264:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acfa7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87e6 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.328280:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.328284:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845031.328289:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acfa7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87e6 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.328304:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:3.0:1788845031.328347:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641558528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9614us (10998us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845031.328355:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 996 00000100:00100000:3.0:1788845031.335025:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.335174:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641558784 00000100:00100000:0.0:1788845031.335198:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641558784 00000100:00100000:0.0:1788845031.335257:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 997 00000100:00100000:0.0:1788845031.335268:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641558784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.335290:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.335311:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acfae lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87f4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.335352:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.335396:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660261120:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.335477:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845031.341360:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.341524:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641558912 00000100:00100000:2.0:1788845031.341534:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641558912 00000100:00100000:2.0:1788845031.341590:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 668 00000100:00100000:2.0:1788845031.341601:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641558912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.341620:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.341641:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfb5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a87fb expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.341711:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660261248:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.341806:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.342112:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660261120:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845031.342135:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845031.342153:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acfae lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87f4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.342173:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.342176:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845031.342181:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287acfae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87f4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.342196:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:0.0:1788845031.342238:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641558784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6972us (7225us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845031.342246:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 997 00000100:00100000:2.0:1788845031.349265:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660261248:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.349308:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfb5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87fb expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.349333:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfb5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87fb expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.349348:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.349425:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641558912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7825us (8074us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.349435:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 668 00000100:00100000:1.0:1788845031.354985:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.355027:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.355407:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641559168 00000100:00100000:2.0:1788845031.355417:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641559168 00000100:00100000:2.0:1788845031.355465:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 669 00000100:00100000:2.0:1788845031.355475:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641559168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.355496:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.355515:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfbc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8809 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.355574:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660261376:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.355667:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.357186:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641559040 00000100:00100000:3.0:1788845031.357200:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641559040 00000100:00100000:3.0:1788845031.357265:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 998 00000100:00100000:3.0:1788845031.357275:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641559040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.357296:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.357317:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acfc3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8802 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.357358:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.357419:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660261504:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845031.357496:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.363233:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660261504:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845031.363261:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845031.363280:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acfc3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8802 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.363302:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.363306:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845031.363311:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acfc3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8802 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.363327:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:3.0:1788845031.363399:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641559040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6116us (8414us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845031.363409:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 998 00000100:00100000:2.0:1788845031.363832:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660261376:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.363856:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfbc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8809 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.363945:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfbc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8809 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.363957:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.363983:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641559168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8511us (8955us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.363990:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 669 00000100:00100000:1.0:1788845031.370072:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.370114:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.370198:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641559296 00000100:00100000:3.0:1788845031.370209:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641559296 00000100:00100000:3.0:1788845031.370288:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 999 00000100:00100000:3.0:1788845031.370299:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641559296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.370320:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.370340:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acfca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8810 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.370379:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.370429:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660261632:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845031.370525:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.375645:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660261632:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845031.375674:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845031.375697:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acfca lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8810 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.375721:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.375724:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845031.375730:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acfca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8810 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.375758:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:3.0:1788845031.375940:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641559296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5642us (5877us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845031.375949:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 999 00000100:00100000:2.0:1788845031.377638:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641559424 00000100:00100000:2.0:1788845031.377651:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641559424 00000100:00100000:2.0:1788845031.377769:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 670 00000100:00100000:2.0:1788845031.377781:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641559424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.377799:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.377817:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfd1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8817 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.377909:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660261760:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.377987:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.380545:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660261760:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.380584:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfd1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8817 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.380609:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfd1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8817 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.380647:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.380768:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641559424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2987us (10651us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.380777:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 670 00000100:00100000:0.0:1788845031.394796:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.394926:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.396923:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641559680 00000100:00100000:2.0:1788845031.396937:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641559680 00000100:00100000:2.0:1788845031.397004:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 671 00000100:00100000:2.0:1788845031.397013:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641559680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.397033:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.397050:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfd8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8825 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.397112:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660261888:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.397197:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.399523:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641559552 00000100:00100000:3.0:1788845031.399537:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641559552 00000100:00100000:2.0:1788845031.400791:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660261888:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.400831:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfd8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8825 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.400857:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfd8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8825 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.401000:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.401064:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641559680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4051us (6135us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.401086:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 671 00000100:00100000:0.0:1788845031.403216:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1000 00000100:00100000:0.0:1788845031.403229:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641559552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.403250:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.403282:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acfdf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a881e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.403318:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.403359:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660262016:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.403454:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.406209:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660262016:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845031.406230:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845031.406245:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acfdf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a881e expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.406411:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.406414:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845031.406419:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287acfdf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a881e expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.406431:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:0.0:1788845031.406479:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641559552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3253us (11690us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845031.406492:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1000 00000100:00100000:1.0:1788845031.410078:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.410166:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.410327:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641559936 00000100:00100000:2.0:1788845031.410338:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641559936 00000100:00100000:2.0:1788845031.410398:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 672 00000100:00100000:2.0:1788845031.410410:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641559936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.410428:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.410445:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfe6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8833 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.410503:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660262144:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.410592:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.412924:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641559808 00000100:00100000:0.0:1788845031.412954:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641559808 00000100:00100000:0.0:1788845031.413151:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1001 00000100:00100000:0.0:1788845031.413164:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641559808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.413184:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.413203:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acfed lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a882c expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.413239:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.413278:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660262272:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.413367:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.426671:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660262144:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.426724:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfe6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8833 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.426748:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acfe6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8833 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.426764:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.426829:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641559936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16419us (16661us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.426837:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 672 00000100:00100000:0.0:1788845031.429147:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660262272:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845031.429183:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845031.429202:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acfed lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a882c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.429222:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.429225:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845031.429230:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287acfed lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a882c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.429245:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:0.0:1788845031.429349:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641559808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16187us (19280us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845031.429357:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1001 00000100:00100000:1.0:1788845031.441661:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.441711:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.442075:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641560192 00000100:00100000:2.0:1788845031.442085:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641560192 00000100:00100000:2.0:1788845031.442159:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 673 00000100:00100000:2.0:1788845031.442169:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641560192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.442189:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.442208:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acff4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8841 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.442265:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660262400:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.442340:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.442573:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641560064 00000100:00100000:3.0:1788845031.442578:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641560064 00000100:00100000:3.0:1788845031.442613:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1002 00000100:00100000:3.0:1788845031.442622:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641560064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.442633:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.442646:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acffb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a883a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.442686:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.442713:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660262528:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845031.442764:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.445648:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660262528:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845031.445675:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845031.445705:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acffb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a883a expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.445725:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.445728:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845031.445734:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287acffb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a883a expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.445758:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00002000:00010000:1.0:1788845031.445927:0:6684:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1337909 -> 1786379 00000100:00100000:1.0:1788845031.445944:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660262400:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845031.445977:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acff4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8841 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.445995:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287acff4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8841 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.446007:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845031.446078:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641560064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3456us (4426us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845031.446087:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1002 00000100:00100000:1.0:1788845031.446485:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641560192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4316us (4772us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845031.446494:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 673 00000100:00100000:1.0:1788845031.454432:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.454482:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.454592:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641560448 00000100:00100000:2.0:1788845031.454605:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641560448 00000100:00100000:2.0:1788845031.454675:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 674 00000100:00100000:2.0:1788845031.454696:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641560448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.454717:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.454737:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2dd021a/0xb2187301287ad002 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a884f expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.454813:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660262656:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.454950:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.455978:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641560320 00000100:00100000:3.0:1788845031.455988:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641560320 00000100:00100000:3.0:1788845031.456026:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1003 00000100:00100000:3.0:1788845031.456033:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641560320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.456219:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.456238:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad009 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8848 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.456273:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.456308:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660262784:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845031.456447:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.475614:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660262656:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845031.475668:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c2dd021a/0xb2187301287ad002 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a884f expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.475706:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2dd021a/0xb2187301287ad002 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a884f expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.475720:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e800, rc 0) 00000100:00100000:3.0:1788845031.482290:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641560448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 27583us (27793us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845031.482303:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 674 00000100:00100000:3.0:1788845031.494579:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660262784:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845031.494611:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845031.494641:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad009 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8848 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.494664:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.494667:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845031.494672:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad009 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8848 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.494699:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845031.494943:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641560320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 38908us (40519us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845031.494952:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1003 00000100:00100000:1.0:1788845031.508551:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.508601:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.508718:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641560576 00000100:00100000:0.0:1788845031.508742:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641560576 00000100:00100000:0.0:1788845031.508815:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1004 00000100:00100000:0.0:1788845031.508826:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641560576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.508848:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.509145:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad010 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8856 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.509188:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.509228:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660262912:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.509301:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.510435:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641560704 00000100:00100000:2.0:1788845031.510446:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641560704 00000100:00100000:2.0:1788845031.510508:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 675 00000100:00100000:2.0:1788845031.510517:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641560704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.510534:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.510551:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ad017 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a885d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.510632:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660263040:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.511157:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.516858:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660262912:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845031.516944:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845031.516965:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad010 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8856 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.516987:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.516990:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845031.516995:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad010 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8856 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.517011:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:0.0:1788845031.517177:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641560576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8351us (8634us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845031.517185:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1004 00000100:00100000:2.0:1788845031.517301:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660263040:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.517325:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ad017 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a885d expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.517343:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287ad017 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a885d expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.517355:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845031.517380:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641560704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6864us (8779us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.517388:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 675 00000100:00100000:3.0:1788845031.520649:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.521087:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641560832 00000100:00100000:0.0:1788845031.521109:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641560832 00000100:00100000:0.0:1788845031.521165:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1005 00000100:00100000:0.0:1788845031.521174:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641560832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.521195:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.521213:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad01e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8864 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.521247:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.521286:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660263168:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.521352:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.536555:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.536688:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641560960 00000100:00100000:1.0:1788845031.536698:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641560960 00000100:00100000:1.0:1788845031.536763:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 676 00000100:00100000:1.0:1788845031.536773:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641560960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845031.536791:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845031.536810:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b774849d/0xb2187301287ad025 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a886b expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845031.537112:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660263296:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845031.537192:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.537976:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660263168:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845031.538069:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845031.538091:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad01e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8864 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.538111:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.538115:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845031.538121:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad01e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8864 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.538135:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:3.0:1788845031.538185:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641560832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17011us (17545us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845031.538193:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1005 00000100:00100000:1.0:1788845031.541164:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660263296:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845031.541223:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b774849d/0xb2187301287ad025 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a886b expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.541248:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b774849d/0xb2187301287ad025 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a886b expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.541263:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b000, rc 0) 00000100:00100000:1.0:1788845031.541328:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641560960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4555us (4781us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845031.541337:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 676 00000100:00100000:0.0:1788845031.545581:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.545857:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641561088 00000100:00100000:3.0:1788845031.545878:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641561088 00000100:00100000:3.0:1788845031.545978:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1006 00000100:00100000:3.0:1788845031.545989:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641561088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.546011:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.546273:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8872 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.546308:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.546319:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.546365:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.546383:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 16 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.546398:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660263424:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845031.546481:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.549279:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660263424:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845031.549316:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.549336:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.549340:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845031.549345:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.549356:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:2.0:1788845031.549470:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641561088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3480us (3896us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845031.549479:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1006 00000100:00100000:1.0:1788845031.555356:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.555721:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641561216 00000100:00100000:2.0:1788845031.555732:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641561216 00010000:00010000:2.0:1788845031.555772:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 17 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.555810:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 359 00000100:00100000:2.0:1788845031.555819:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641561216:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845031.555832:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845031.555857:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 17 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.555925:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 17 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.555948:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287acf76 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87ca expref: 17 pid: 5840 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.555962:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.555968:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.555978:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.556003:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.556051:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.556064:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660263552:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845031.556138:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.561969:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.562194:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641561344 00000100:00100000:1.0:1788845031.562204:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641561344 00000100:00100000:1.0:1788845031.562280:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 654 00000100:00100000:1.0:1788845031.562292:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641561344:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845031.562535:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788845031.562540:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845031 -> 1788845031 00000004:00010000:1.0:1788845031.562542:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:1.0:1788845031.562545:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845031 -> 1788845031 00000004:00010000:1.0:1788845031.562547:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845031.562631:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641561344:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 341us (673us total) trans 4294967678 rc 0/0 00000100:00100000:1.0:1788845031.562641:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 654 00000100:00100000:2.0:1788845031.563352:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660263552:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845031.563377:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845031.563410:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641561216:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7592us (8064us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845031.563420:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 359 00000100:00100000:0.0:1788845031.573600:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.573716:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641561472 00000100:00100000:1.0:1788845031.573728:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641561472 00000100:00100000:1.0:1788845031.573809:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 677 00000100:00100000:1.0:1788845031.573820:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641561472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845031.573840:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845031.573859:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000054eb58a4/0xb2187301287ad033 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8879 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.574336:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660263680:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.574436:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.577724:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660263680:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845031.577768:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000054eb58a4/0xb2187301287ad033 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8879 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.577795:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054eb58a4/0xb2187301287ad033 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8879 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.577810:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1a000, rc 0) 00000100:00100000:0.0:1788845031.578007:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641561472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4187us (4415us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845031.578018:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 677 00000100:00100000:3.0:1788845031.584302:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.584455:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641561600 00000100:00100000:0.0:1788845031.584472:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641561600 00000100:00100000:0.0:1788845031.584564:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 678 00000100:00100000:0.0:1788845031.584577:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641561600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.584596:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.584615:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad03a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8880 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.584686:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660263808:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.584784:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.589610:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660263808:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845031.589654:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad03a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8880 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.589692:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad03a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8880 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.589708:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030200, rc 0) 00000100:00100000:0.0:1788845031.589784:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641561600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5210us (5498us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845031.589794:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 678 00000100:00100000:2.0:1788845031.600821:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.601020:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641561728 00000100:00100000:3.0:1788845031.601029:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641561728 00000100:00100000:3.0:1788845031.601109:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1007 00000100:00100000:3.0:1788845031.601118:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641561728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.601142:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.601165:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad041 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8887 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.601204:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.601249:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660263936:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845031.601329:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.607492:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660263936:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845031.607520:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788845031.607542:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad041 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8887 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.607639:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.607642:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845031.607648:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad041 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8887 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.607800:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:2.0:1788845031.608211:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641561728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7092us (7402us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.608221:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1007 00000100:00100000:2.0:1788845031.613958:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.614003:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641561856 00000100:00100000:1.0:1788845031.614013:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641561856 00000100:00100000:1.0:1788845031.614083:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1008 00000100:00100000:1.0:1788845031.614094:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641561856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845031.614115:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845031.614137:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a888e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.614170:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.614180:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a888e expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.614215:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.614233:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 11 pid: 5841 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845031.614248:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660264064:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845031.614358:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845031.629312:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660264064:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845031.629359:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a888e expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.629381:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845031.629384:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845031.629390:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a888e expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.629403:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ae00, rc 0) 00000100:00100000:1.0:1788845031.629529:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641561856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15437us (15581us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845031.629538:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1008 00000100:00100000:3.0:1788845031.634096:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.634185:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641561984 00000100:00100000:0.0:1788845031.634210:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641561984 00010000:00010000:0.0:1788845031.634250:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 12 pid: 5841 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.634295:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 360 00000100:00100000:0.0:1788845031.634306:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641561984:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845031.634321:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845031.634350:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 12 pid: 5841 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.634370:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 12 pid: 5841 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.634465:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad02c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8872 expref: 12 pid: 5841 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.634483:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845031.634490:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a888e expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.634502:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a888e expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.634511:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a888e expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.634731:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a888e expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.634825:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660264192:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845031.637946:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845031.640257:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660264192:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845031.640284:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845031.640344:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641561984:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6039us (6258us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845031.640355:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 360 00000100:00100000:1.0:1788845031.641510:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.641835:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641562112 00000100:00100000:3.0:1788845031.641845:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641562112 00002000:00100000:3.0:1788845031.642001:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48000 x1875739641562112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845031.642013:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48000 with x1875739641562112 00000100:00100000:3.0:1788845031.642079:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 655 00000100:00100000:3.0:1788845031.642089:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641562112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845031.642302:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641562112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 214us (795us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845031.642311:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 655 00002000:00100000:3.0:1788845031.642317:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48000 x1875739641562112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845031.642325:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48000 with x1875739641562112 00000100:00100000:3.0:1788845031.649074:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.649176:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641562240 00000100:00100000:0.0:1788845031.649187:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641562240 00002000:00100000:0.0:1788845031.649245:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1339392->1748991) req@ffff94a035742680 x1875739641562240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.649257:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035742680 with x1875739641562240 ext(1339392->1748991) 00002000:00010000:0.0:1788845031.649264:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035742680 00000100:00100000:0.0:1788845031.649301:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 373 00000100:00100000:0.0:1788845031.649311:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641562240:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845031.659770:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.660808:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641562368 00000100:00100000:2.0:1788845031.660821:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641562368 00002000:00100000:2.0:1788845031.660934:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8f800 x1875739641562368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.660947:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8f800 with x1875739641562368 00000100:00100000:2.0:1788845031.660985:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 656 00000100:00100000:2.0:1788845031.660995:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641562368:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845031.662009:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641562240:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12693us (12940us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845031.662020:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 373 00002000:00100000:0.0:1788845031.662031:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1339392->1748991) req@ffff94a035742680 x1875739641562240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/440 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.662040:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035742680 with x1875739641562240 ext(1339392->1748991) 00002000:00010000:0.0:1788845031.662045:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035742680 00000100:00100000:2.0:1788845031.667739:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641562368:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 6732us (7977us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845031.667763:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 656 00002000:00100000:2.0:1788845031.667779:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8f800 x1875739641562368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.667790:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8f800 with x1875739641562368 00000100:00100000:1.0:1788845031.674695:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.674913:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641562496 00000100:00100000:0.0:1788845031.674923:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641562496 00002000:00100000:0.0:1788845031.674972:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8fb80 x1875739641562496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.674982:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8fb80 with x1875739641562496 00000100:00100000:0.0:1788845031.675020:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 657 00000100:00100000:0.0:1788845031.675030:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641562496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845031.675282:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641562496:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 252us (596us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845031.675289:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 657 00002000:00100000:0.0:1788845031.675296:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8fb80 x1875739641562496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.675304:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8fb80 with x1875739641562496 00000100:00100000:1.0:1788845031.687334:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.687770:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641562624 00000100:00100000:0.0:1788845031.687781:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641562624 00002000:00100000:0.0:1788845031.687831:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8ce00 x1875739641562624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.687843:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8ce00 with x1875739641562624 00000100:00100000:0.0:1788845031.687928:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 658 00000100:00100000:0.0:1788845031.687939:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641562624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845031.688159:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641562624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 221us (834us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845031.688167:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 658 00002000:00100000:0.0:1788845031.688174:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8ce00 x1875739641562624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.688182:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8ce00 with x1875739641562624 00000100:00100000:2.0:1788845031.697355:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.697458:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641562752 00000100:00100000:0.0:1788845031.697468:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641562752 00002000:00100000:0.0:1788845031.697517:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364a80 x1875739641562752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.697527:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364a80 with x1875739641562752 00000100:00100000:0.0:1788845031.697690:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 659 00000100:00100000:0.0:1788845031.697702:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641562752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845031.698209:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641562752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 507us (862us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845031.698227:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 659 00002000:00100000:1.0:1788845031.698235:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364a80 x1875739641562752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845031.698244:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364a80 with x1875739641562752 00000100:00100000:1.0:1788845031.718036:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.718211:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641562880 00000100:00100000:2.0:1788845031.718222:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641562880 00002000:00100000:2.0:1788845031.718271:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8f100 x1875739641562880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.718284:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8f100 with x1875739641562880 00000100:00100000:2.0:1788845031.718325:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 660 00000100:00100000:2.0:1788845031.718334:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641562880:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845031.718596:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641562880:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 262us (568us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845031.718604:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 660 00002000:00100000:2.0:1788845031.718612:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8f100 x1875739641562880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.718620:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8f100 with x1875739641562880 00000100:00100000:0.0:1788845031.731133:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.731215:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641563008 00000100:00100000:3.0:1788845031.731228:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641563008 00002000:00100000:3.0:1788845031.731288:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc4700 x1875739641563008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845031.731300:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4700 with x1875739641563008 00000100:00100000:3.0:1788845031.731380:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 661 00000100:00100000:3.0:1788845031.731391:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641563008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845031.731635:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641563008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 244us (511us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845031.731642:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 661 00002000:00100000:3.0:1788845031.731659:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc4700 x1875739641563008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845031.731687:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4700 with x1875739641563008 00000100:00100000:2.0:1788845031.743168:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.743280:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641563136 00000100:00100000:0.0:1788845031.743291:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641563136 00002000:00100000:0.0:1788845031.743350:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364380 x1875739641563136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.743362:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364380 with x1875739641563136 00000100:00100000:0.0:1788845031.743408:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 662 00000100:00100000:0.0:1788845031.743418:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641563136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845031.743917:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641563136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 463us (722us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845031.743927:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 662 00002000:00100000:0.0:1788845031.743934:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364380 x1875739641563136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.743943:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364380 with x1875739641563136 00000100:00100000:0.0:1788845031.753820:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.754758:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641563264 00000100:00100000:1.0:1788845031.754771:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641563264 00002000:00100000:1.0:1788845031.754823:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5880 x1875739641563264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845031.754835:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5880 with x1875739641563264 00000100:00100000:1.0:1788845031.754924:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 663 00000100:00100000:1.0:1788845031.754935:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641563264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845031.755216:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641563264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 282us (1405us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845031.755225:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 663 00002000:00100000:1.0:1788845031.755233:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5880 x1875739641563264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845031.755242:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5880 with x1875739641563264 00000100:00100000:2.0:1788845031.766663:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.766790:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641563392 00000100:00100000:0.0:1788845031.766803:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641563392 00002000:00100000:0.0:1788845031.766860:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364000 x1875739641563392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.766920:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364000 with x1875739641563392 00000100:00100000:0.0:1788845031.766945:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 664 00000100:00100000:0.0:1788845031.766956:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641563392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845031.767246:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641563392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 291us (594us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845031.767255:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 664 00002000:00100000:0.0:1788845031.767262:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364000 x1875739641563392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845031.767270:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364000 with x1875739641563392 00000100:00100000:2.0:1788845031.780921:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.784250:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641563520 00000100:00100000:1.0:1788845031.784267:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641563520 00002000:00100000:1.0:1788845031.784330:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02365180 x1875739641563520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845031.784343:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02365180 with x1875739641563520 00000100:00100000:1.0:1788845031.784391:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 665 00000100:00100000:1.0:1788845031.784418:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641563520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845031.784715:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641563520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 298us (3854us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845031.784724:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 665 00002000:00100000:1.0:1788845031.784731:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02365180 x1875739641563520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845031.784739:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02365180 with x1875739641563520 00000100:00100000:1.0:1788845031.795748:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.795996:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641563648 00000100:00100000:2.0:1788845031.796006:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641563648 00002000:00100000:2.0:1788845031.796061:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8df80 x1875739641563648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.796073:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8df80 with x1875739641563648 00000100:00100000:2.0:1788845031.796107:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 666 00000100:00100000:2.0:1788845031.796118:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641563648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845031.796376:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641563648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (647us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845031.796384:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 666 00002000:00100000:2.0:1788845031.796392:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8df80 x1875739641563648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845031.796400:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8df80 with x1875739641563648 00000100:00100000:2.0:1788845031.807922:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.808020:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641563776 00000100:00100000:1.0:1788845031.808029:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641563776 00002000:00100000:1.0:1788845031.808080:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9a680 x1875739641563776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/0 e 0 to 0 dl 1788845042 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845031.808093:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9a680 with x1875739641563776 00000100:00100000:1.0:1788845031.808139:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 667 00000100:00100000:1.0:1788845031.808149:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641563776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845031.808371:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641563776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 223us (509us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845031.808378:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 667 00002000:00100000:1.0:1788845031.808384:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9a680 x1875739641563776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:137/0 lens 488/4536 e 0 to 0 dl 1788845042 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845031.808392:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9a680 with x1875739641563776 00000100:00100000:3.0:1788845031.822252:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.825762:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641564032 00000100:00100000:1.0:1788845031.825778:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641564032 00000100:00100000:1.0:1788845031.825860:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1009 00000100:00100000:1.0:1788845031.825938:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641564032:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845031.826008:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845031.826039:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287acf8b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d8 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.826061:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845031.826065:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845031.826274:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.826290:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.826299:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.826334:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.826352:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 10 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845031.826368:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8f480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660264320:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845031.826471:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.831005:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.831944:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8f480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660264320:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845031.831978:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.832138:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641564160 00000100:00100000:2.0:1788845031.832164:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641564160 00010000:00010000:2.0:1788845031.832203:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 11 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.832247:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 361 00000100:00100000:2.0:1788845031.832256:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641564160:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845031.832272:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845031.832279:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 11 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.832298:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 11 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.832320:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287acf99 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87df expref: 11 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.832336:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.832341:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.832351:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.832358:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.832387:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845031.832452:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641564160:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 196us (1462us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845031.832471:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 361 00010000:00010000:3.0:1788845031.832502:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.832518:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845031.832707:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967679 00000004:00010000:3.0:1788845031.832776:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845031.832781:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845031.832783:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845031.832786:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845031.832788:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:3.0:1788845031.832798:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c1bdea47 reply state 0000000044feb866 transno 4294967680 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.832810:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845031.832987:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641564032:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 7103us (10749us total) trans 4294967680 rc 0/0 00000100:00100000:3.0:1788845031.832996:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1009 00000020:00080000:3.0:1788845031.833002:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845031.846059:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.846099:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.846107:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.846113:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01e1ac00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845031.846118:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.846126:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.846145:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845031.846148:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845031.846153:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.846159:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad04f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.851227:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.857273:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641564288 00000100:00100000:2.0:1788845031.857288:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641564288 00000100:00100000:2.0:1788845031.857361:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1010 00000100:00100000:2.0:1788845031.857372:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641564288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.857398:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.857401:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845031.857447:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad048 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a888e expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.857467:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.857472:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845031.857488:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8895 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.857513:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.857526:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.857536:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.857539:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845031.857541:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.857624:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641564288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 252us (6414us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845031.857649:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1010 00000100:00100000:3.0:1788845031.865521:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.865712:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641564416 00000100:00100000:2.0:1788845031.865748:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641564416 00002000:00010000:2.0:1788845031.865797:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845031.865803:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742d80 with x1875739641564416 00000100:00100000:2.0:1788845031.865839:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 668 00000100:00100000:2.0:1788845031.865849:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641564416:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:2.0:1788845031.866216:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 683971 00000004:00010000:2.0:1788845031.866220:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845031 -> 1788845031 00000004:00010000:2.0:1788845031.866223:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:2.0:1788845031.866226:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845031 -> 1788845031 00000004:00010000:2.0:1788845031.866227:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:2.0:1788845031.866306:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641564416:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 457us (801us total) trans 4294967681 rc 0/0 00000100:00100000:2.0:1788845031.866314:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 668 00002000:00010000:2.0:1788845031.866318:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845031.866320:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742d80 with x1875739641564416 00000100:00100000:3.0:1788845031.867629:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.867762:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641564544 00000100:00100000:2.0:1788845031.867773:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641564544 00002000:00010000:2.0:1788845031.867824:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845031.867829:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035740a80 with x1875739641564544 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845031.867843:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035740a80. 00000100:00100000:2.0:1788845031.867921:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 374 00000100:00100000:2.0:1788845031.867932:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641564544:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845031.868331:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1786379 -> 0 00002000:00010000:0.0:1788845031.868340:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845031 -> 1788845031 00002000:00010000:0.0:1788845031.868343:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845031 -> 1788845029 00002000:00010000:0.0:1788845031.868345:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845031 -> 1788845031 00002000:00010000:0.0:1788845031.868347:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 15 -> 15 00000100:00100000:0.0:1788845031.868423:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641564544:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 489us (799us total) trans 4294967422 rc 0/0 00000100:00100000:0.0:1788845031.868434:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 374 00002000:00010000:0.0:1788845031.868439:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845031.868443:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035740a80 with x1875739641564544 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845031.868450:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035740a80. 00000100:00100000:1.0:1788845031.880141:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.881705:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641564672 00000100:00100000:2.0:1788845031.881733:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641564672 00000100:00100000:2.0:1788845031.881802:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1011 00000100:00100000:2.0:1788845031.881814:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641564672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.881843:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.881930:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ad05d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a889c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.882165:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad064 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.882210:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad064 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.882226:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad064 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845031.882262:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 683971 00000004:00200000:3.0:1788845031.882268:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845031.882291:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad064 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.882332:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ad05d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a889c expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.882361:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad064 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a889c expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.882371:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.882374:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845031.882377:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e200, rc 0) 00000100:00100000:3.0:1788845031.882619:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641564672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 805us (2485us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845031.882629:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1011 00000100:00100000:3.0:1788845031.895854:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.895962:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.897066:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641564800 00000100:00100000:2.0:1788845031.897082:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641564800 00000100:00100000:2.0:1788845031.897168:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1012 00000100:00100000:2.0:1788845031.897180:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641564800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.897201:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.897223:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ad06b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88a3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.897270:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.897318:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660264448:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.897416:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.897520:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641564928 00000100:00100000:0.0:1788845031.897525:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641564928 00000100:00100000:0.0:1788845031.897582:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 679 00000100:00100000:0.0:1788845031.897590:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641564928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.897600:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.897612:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad072 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88aa expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.897667:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660264576:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845031.897695:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.903160:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660264448:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845031.903200:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:2.0:1788845031.903227:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ad06b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88a3 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.903253:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.903256:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845031.903286:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ad06b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88a3 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.903307:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845031.903401:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641564800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6222us (7561us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.903411:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1012 00002000:00010000:0.0:1788845031.913643:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845031 00000100:00100000:0.0:1788845031.914014:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660264576:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845031.914072:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad072 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88aa expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.914930:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad072 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88aa expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.914953:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030200, rc 0) 00000100:00100000:1.0:1788845031.915277:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641564928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17684us (19311us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845031.915289:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 679 00000100:00100000:0.0:1788845031.923296:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.923359:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845031.923383:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641565056 00000100:00100000:3.0:1788845031.923400:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641565056 00000100:00100000:3.0:1788845031.923486:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1013 00000100:00100000:3.0:1788845031.923498:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641565056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845031.923546:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845031.923572:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287ad079 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88b1 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.923626:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845031.923659:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641565184 00000100:00100000:1.0:1788845031.923667:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641565184 00000100:00100000:3.0:1788845031.923680:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035740700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660264704:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845031.923728:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 680 00000100:00100000:1.0:1788845031.923738:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641565184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845031.923759:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845031.923774:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006b387054/0xb2187301287ad080 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88b8 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845031.923789:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845031.923824:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660264832:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845031.923856:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.930437:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660264832:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.930502:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006b387054/0xb2187301287ad080 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88b8 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.930539:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b387054/0xb2187301287ad080 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88b8 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.930558:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b800, rc 0) 00000100:00100000:2.0:1788845031.930645:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641565184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6908us (7285us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.930657:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 680 00000100:00100000:1.0:1788845031.931607:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035740700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660264704:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845031.931634:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:1.0:1788845031.931654:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287ad079 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88b1 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.931674:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845031.931677:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845031.931683:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287ad079 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88b1 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.931711:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:1.0:1788845031.931783:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641565056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8286us (8504us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845031.931792:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1013 00000100:00100000:3.0:1788845031.938622:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.938995:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641565312 00000100:00100000:1.0:1788845031.939009:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641565312 00000100:00100000:1.0:1788845031.939132:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1014 00000100:00100000:1.0:1788845031.939145:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641565312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845031.939191:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845031.939221:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad087 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88bf expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.939359:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.939378:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.939391:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845031.939426:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 683971 00000004:00200000:1.0:1788845031.939431:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845031.939454:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.939489:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ae00/0xb2187301287ad087 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.939509:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.939519:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845031.939522:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845031.939526:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1be00, rc 0) 00000100:00100000:1.0:1788845031.939626:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641565312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 483us (1098us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845031.939633:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1014 00000100:00100000:2.0:1788845031.949076:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845031.949325:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641565440 00000100:00100000:0.0:1788845031.949340:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641565440 00000100:00100000:0.0:1788845031.949463:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1015 00000100:00100000:0.0:1788845031.949476:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641565440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845031.949510:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845031.949540:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88c6 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.949591:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.949627:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.949677:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845031.949700:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 19 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845031.949719:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660264960:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845031.949957:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845031.953383:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660264960:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845031.953437:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.953464:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845031.953467:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845031.953472:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845031.953486:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:3.0:1788845031.953850:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641565440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4375us (4790us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845031.953872:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1015 00000100:00100000:3.0:1788845031.960018:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.960121:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641565568 00000100:00100000:2.0:1788845031.960135:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641565568 00010000:00010000:2.0:1788845031.960365:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 20 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.960461:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 362 00000100:00100000:2.0:1788845031.960473:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d4b050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641565568:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845031.960520:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845031.960570:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 20 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.960599:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 20 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.960630:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad056 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8895 expref: 20 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.960657:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845031.960664:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.960677:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.960686:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.960736:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.960754:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660265088:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845031.960877:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845031.968488:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.968664:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641565696 00000100:00100000:1.0:1788845031.968676:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641565696 00000100:00100000:1.0:1788845031.968794:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 669 00000100:00100000:1.0:1788845031.968808:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641565696:12345-192.168.201.29@tcp:22:fsx.0 00000100:00100000:2.0:1788845031.969008:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660265088:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845031.969064:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00010000:1.0:1788845031.969231:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 683971 00000004:00010000:1.0:1788845031.969236:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845031 -> 1788845031 00000004:00010000:1.0:1788845031.969239:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:1.0:1788845031.969242:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845031 -> 1788845031 00000004:00010000:1.0:1788845031.969245:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845031.969340:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641565696:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 535us (865us total) trans 4294967682 rc 0/0 00000100:00100000:1.0:1788845031.969354:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 669 00000100:00100000:0.0:1788845031.969566:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d4b050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641565568:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9094us (9562us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845031.969577:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 362 00000100:00100000:0.0:1788845031.972992:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.973961:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641565824 00000100:00100000:2.0:1788845031.973978:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641565824 00000100:00100000:2.0:1788845031.974114:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 681 00000100:00100000:2.0:1788845031.974128:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641565824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.974164:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.974192:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad09c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88cd expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.974279:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660265216:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.974407:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.977834:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660265216:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.977954:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad09c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88cd expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.977988:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad09c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88cd expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.978008:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845031.978082:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641565824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3956us (5103us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.978091:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 681 00000100:00100000:0.0:1788845031.982967:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845031.983058:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641565952 00000100:00100000:2.0:1788845031.983072:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641565952 00000100:00100000:2.0:1788845031.983143:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 682 00000100:00100000:2.0:1788845031.983156:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641565952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845031.983202:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845031.983228:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad0a3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88d4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845031.983309:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660265344:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845031.983477:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845031.989143:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660265344:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845031.989202:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad0a3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88d4 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.989235:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad0a3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88d4 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845031.989252:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:2.0:1788845031.989332:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641565952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6177us (6377us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845031.989343:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 682 00000100:00100000:3.0:1788845031.993607:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845031.993851:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641566080 00000100:00100000:1.0:1788845031.993874:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641566080 00000100:00100000:1.0:1788845031.994107:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 683 00000100:00100000:1.0:1788845031.994120:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641566080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845031.994147:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845031.994179:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a463c78/0xb2187301287ad0aa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88db expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845031.994263:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660265472:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845031.994363:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f480 going to sleep for 11 seconds 00000100:00100000:1.0:1788845031.997185:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660265472:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845031.997243:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a463c78/0xb2187301287ad0aa lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88db expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.997277:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a463c78/0xb2187301287ad0aa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88db expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845031.997315:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b200, rc 0) 00000100:00100000:1.0:1788845031.997436:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641566080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3317us (3844us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845031.997446:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 683 00000100:00100000:0.0:1788845032.000730:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.000846:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641566208 00000100:00100000:1.0:1788845032.000871:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641566208 00000100:00100000:1.0:1788845032.000976:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 684 00000100:00100000:1.0:1788845032.000990:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641566208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845032.001018:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845032.001044:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094f5438e/0xb2187301287ad0b1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88e2 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845032.001127:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660265600:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845032.001243:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f480 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.004452:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660265600:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845032.004510:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000094f5438e/0xb2187301287ad0b1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e2 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.004543:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094f5438e/0xb2187301287ad0b1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e2 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.004560:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1bc00, rc 0) 00000100:00100000:1.0:1788845032.004635:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641566208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3648us (3919us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845032.004666:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 684 00000100:00100000:0.0:1788845032.012406:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.012794:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641566336 00000100:00100000:2.0:1788845032.012809:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641566336 00000100:00100000:0.0:1788845032.013205:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 685 00000100:00100000:0.0:1788845032.013218:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641566336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.013247:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.013286:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88e9 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.013358:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.013369:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(1 11) 00010000:00010000:0.0:1788845032.013374:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:0.0:1788845032.013380:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1642496->1654783] (req 1642496->1654783) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.013429:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 7 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.013454:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 7 pid: 6682 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.013473:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660265728:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845032.013627:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845032.017401:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660265728:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845032.017458:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1642496->1654783] (req 1642496->1654783) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.017491:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030200, rc 0) 00000100:00100000:0.0:1788845032.017633:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641566336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4417us (5239us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845032.017670:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 685 00010000:00010000:3.0:1788845032.018731:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845032.030601:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.031247:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641566464 00000100:00100000:2.0:1788845032.031261:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641566464 00010000:00010000:2.0:1788845032.031320:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 8 pid: 6682 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.031419:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 363 00000100:00100000:2.0:1788845032.031433:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641566464:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845032.031481:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845032.031515:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 8 pid: 6682 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.031538:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 8 pid: 6682 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.031569:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287acf7d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a87d1 expref: 8 pid: 6682 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.031591:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788845032.031596:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788845032.031606:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.031619:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.031678:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.031698:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660265856:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845032.031833:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.041037:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660265856:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845032.041076:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845032.041191:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641566464:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9759us (10607us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.041206:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 363 00000100:00100000:3.0:1788845032.042456:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.042501:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.042591:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641566720 00000100:00100000:0.0:1788845032.042604:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641566720 00000100:00100000:0.0:1788845032.042679:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 686 00000100:00100000:0.0:1788845032.042692:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641566720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.042717:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.042750:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b79c20b/0xb2187301287ad0bf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88f7 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.042833:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660265984:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.042982:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.043189:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641566592 00000100:00100000:2.0:1788845032.043195:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641566592 00000100:00100000:2.0:1788845032.043244:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1016 00000100:00100000:2.0:1788845032.043250:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641566592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.043261:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.043275:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0c6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88f0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.043305:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.043325:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660266112:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.043349:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00002000:00010000:0.0:1788845032.049695:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1651231 00002000:00010000:0.0:1788845032.049710:0:6682:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845031 -> 1788845032 00002000:00010000:0.0:1788845032.049713:0:6682:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845031 -> 1788845032 00000100:00100000:0.0:1788845032.049753:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660265984:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845032.049809:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b79c20b/0xb2187301287ad0bf lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88f7 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.049841:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b79c20b/0xb2187301287ad0bf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88f7 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.049869:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:0.0:1788845032.049995:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641566720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7306us (7547us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.050028:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 686 00000004:00010000:2.0:1788845032.050625:0:5841:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845031 -> 1788845032 00000004:00010000:2.0:1788845032.050633:0:5841:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845031 -> 1788845032 00000100:00100000:2.0:1788845032.050658:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660266112:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845032.050680:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:2.0:1788845032.050697:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0c6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88f0 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.050715:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.050718:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845032.050724:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88f0 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.050768:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845032.050805:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641566592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7555us (8303us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.050813:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1016 00000100:00100000:3.0:1788845032.057414:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.059851:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.060300:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641566976 00000100:00100000:2.0:1788845032.060315:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641566976 00000100:00100000:2.0:1788845032.060395:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 687 00000100:00100000:2.0:1788845032.060408:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641566976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.060435:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.060465:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ad0cd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8905 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.060600:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660266240:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.060721:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.065495:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660266240:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.065552:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ad0cd lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8905 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.065586:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287ad0cd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8905 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.065603:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845032.065687:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641566976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5280us (5850us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.065698:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 687 00000100:00100000:1.0:1788845032.067783:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641566848 00000100:00100000:1.0:1788845032.067802:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641566848 00000100:00100000:2.0:1788845032.070106:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1017 00000100:00100000:2.0:1788845032.070146:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2367:x1875739641566848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.070188:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.070219:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0d4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a88fe expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.070275:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.070335:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660266368:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.070463:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.073623:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660266368:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845032.073662:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:2.0:1788845032.073686:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0d4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88fe expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.073713:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.073716:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845032.073721:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88fe expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.073814:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845032.073944:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2367:x1875739641566848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3802us (16545us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.073953:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1017 00000100:00100000:3.0:1788845032.079173:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.079286:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.079416:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641567104 00000100:00100000:0.0:1788845032.079430:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641567104 00000100:00100000:0.0:1788845032.079504:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1018 00000100:00100000:0.0:1788845032.079517:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641567104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.079665:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.079693:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad0db lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a890c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.079755:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.079807:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660266496:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.080109:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.080235:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641567232 00000100:00100000:1.0:1788845032.080240:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641567232 00000100:00100000:1.0:1788845032.080336:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 688 00000100:00100000:1.0:1788845032.080346:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641567232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845032.080357:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845032.080371:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000020e58b6a/0xb2187301287ad0e2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8913 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845032.080429:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660266624:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845032.080464:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f480 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.084837:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660266624:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.084939:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000020e58b6a/0xb2187301287ad0e2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8913 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.084993:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000020e58b6a/0xb2187301287ad0e2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8913 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.085011:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1a600, rc 0) 00000100:00100000:2.0:1788845032.085098:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641567232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4753us (6041us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.085108:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 688 00000100:00100000:1.0:1788845032.085728:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660266496:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845032.085763:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:1.0:1788845032.085780:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad0db lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a890c expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.085797:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845032.085800:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845032.085805:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad0db lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a890c expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.085821:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:1.0:1788845032.085877:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641567104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6362us (6591us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845032.085944:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1018 00000100:00100000:0.0:1788845032.093370:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.093444:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.093653:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641567488 00000100:00100000:2.0:1788845032.093668:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641567488 00000100:00100000:2.0:1788845032.093753:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 689 00000100:00100000:2.0:1788845032.093765:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641567488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.093790:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.093816:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad0e9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8921 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.093948:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660266752:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.094057:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.095365:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641567360 00000100:00100000:3.0:1788845032.095378:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641567360 00000100:00100000:3.0:1788845032.095459:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1019 00000100:00100000:3.0:1788845032.095469:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641567360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.095494:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.095516:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ad0f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a891a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.095572:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.095617:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660266880:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.095708:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.100193:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660266880:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845032.100233:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:3.0:1788845032.100272:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ad0f0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a891a expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.100298:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.100300:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845032.100306:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ad0f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a891a expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.100352:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:3.0:1788845032.100439:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641567360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4970us (6992us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.100450:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1019 00000100:00100000:2.0:1788845032.102833:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660266752:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.102940:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad0e9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8921 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.102972:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad0e9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8921 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.103015:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845032.103242:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641567488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9478us (9885us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.103252:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 689 00000100:00100000:1.0:1788845032.109334:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.109426:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641567744 00000100:00100000:2.0:1788845032.109441:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641567744 00000100:00100000:2.0:1788845032.109518:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1020 00000100:00100000:2.0:1788845032.109530:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641567744:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845032.109627:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845032.109664:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad064 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a889c expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.109687:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.109692:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845032.110078:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.110096:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.110107:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.110157:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.110213:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 18 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.110236:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660267008:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845032.110359:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.114145:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660267008:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845032.114192:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845032.116276:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.116519:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641567872 00000100:00100000:0.0:1788845032.116545:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641567872 00010000:00010000:0.0:1788845032.116604:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 19 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.116660:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 364 00000100:00100000:0.0:1788845032.116672:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641567872:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845032.116690:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845032.116721:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 19 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.116755:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 19 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.116783:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1be00/0xb2187301287ad08e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88bf expref: 19 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.116803:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.116809:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.116823:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.116832:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.116879:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845032.116982:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641567872:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 310us (728us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845032.116990:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 364 00010000:00010000:1.0:1788845032.117924:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.117939:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845032.118108:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967683 00000004:00010000:1.0:1788845032.118180:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 683971 00000004:00010000:1.0:1788845032.118184:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845032 -> 1788845032 00000004:00010000:1.0:1788845032.118187:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:1.0:1788845032.118189:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845032 -> 1788845032 00000004:00010000:1.0:1788845032.118191:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:1.0:1788845032.118199:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d2052642 reply state 000000007f4f50bf transno 4294967684 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845032.118208:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845032.118347:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641567744:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8815us (9025us total) trans 4294967684 rc 0/0 00000100:00100000:1.0:1788845032.118357:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1020 00000020:00080000:1.0:1788845032.118364:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845032.121924:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.121947:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.121955:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.121961:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216fa00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845032.121967:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.121976:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.121996:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.121999:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845032.122004:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.122029:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0f7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.125063:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.125176:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641568000 00000100:00100000:3.0:1788845032.125189:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641568000 00002000:00010000:3.0:1788845032.125246:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1187877->18446744073709551615) 00002000:00100000:3.0:1788845032.125252:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359b8e00 with x1875739641568000 ext(1187877->18446744073709551615) 00002000:00010000:3.0:1788845032.125279:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359b8e00. 00000100:00100000:3.0:1788845032.125317:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 375 00000100:00100000:3.0:1788845032.125329:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641568000:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845032.125631:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1651231 -> 1187877 00002000:00010000:3.0:1788845032.125635:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845032 -> 1788845032 00002000:00010000:3.0:1788845032.125638:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845031 -> 1788845029 00002000:00010000:3.0:1788845032.125641:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845032 -> 1788845032 00002000:00010000:3.0:1788845032.125643:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 15 -> 15 00000100:00100000:3.0:1788845032.125715:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641568000:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 387us (665us total) trans 4294967423 rc 0/0 00000100:00100000:3.0:1788845032.125722:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 375 00002000:00010000:3.0:1788845032.125727:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1187877->18446744073709551615) 00002000:00100000:3.0:1788845032.125730:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359b8e00 with x1875739641568000 ext(1187877->18446744073709551615) 00002000:00010000:3.0:1788845032.125735:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359b8e00. 00000100:00100000:1.0:1788845032.131294:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.131428:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641568128 00000100:00100000:2.0:1788845032.131443:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641568128 00000100:00100000:2.0:1788845032.131526:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1021 00000100:00100000:2.0:1788845032.131613:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641568128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.131652:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.131682:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8928 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.132039:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ad105 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.132060:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ad105 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.132077:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ad105 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845032.132114:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1187877 00000004:00200000:2.0:1788845032.132118:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845032.132141:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ad105 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.132175:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad0fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8928 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.132201:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ad105 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8928 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.132228:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.132231:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845032.132235:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:2.0:1788845032.132370:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641568128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 762us (1090us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.132377:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1021 00000100:00100000:1.0:1788845032.142227:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.142485:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641568256 00000100:00100000:0.0:1788845032.142499:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641568256 00000100:00100000:0.0:1788845032.142693:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1022 00000100:00100000:0.0:1788845032.142707:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641568256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.142747:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.142796:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287ad10c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a892f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.142971:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.142991:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.143005:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845032.143039:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1187877 00000004:00200000:0.0:1788845032.143044:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845032.143069:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.143101:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287ad10c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.143121:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.143131:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.143135:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845032.143138:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031200, rc 0) 00000100:00100000:0.0:1788845032.143223:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641568256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 519us (1013us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845032.143233:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1022 00000100:00100000:1.0:1788845032.149667:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660267136:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788845032.149784:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845032.150152:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660267136 00000100:00100000:2.0:1788845032.150164:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660267136 00000100:00100000:2.0:1788845032.150248:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 80 00000100:00100000:2.0:1788845032.150261:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660267136:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1788845032.150310:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1788845032.150368:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660267136:12345-0@lo:13:osp-pre-0-0.0 Request processed in 110us (584us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.150375:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 80 00000100:00080000:1.0:1788845032.150554:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845032.150601:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660267136:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845032.153782:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.153908:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.153994:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641568384 00000100:00100000:3.0:1788845032.154007:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641568384 00000100:00100000:3.0:1788845032.154086:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1023 00000100:00100000:3.0:1788845032.154097:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641568384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.154127:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.154153:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ad11a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8936 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.154204:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.154258:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660267264:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.154359:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.154545:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641568512 00000100:00100000:2.0:1788845032.154550:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641568512 00000100:00100000:2.0:1788845032.154592:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 690 00000100:00100000:2.0:1788845032.154600:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641568512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.154609:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.154623:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad121 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a893d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.154664:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660267392:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.154691:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845032.159667:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660267264:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845032.159710:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:0.0:1788845032.159749:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ad11a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8936 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.159779:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.159783:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845032.159789:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f000/0xb2187301287ad11a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8936 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.159810:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:0.0:1788845032.159973:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641568384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5876us (6206us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.159984:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1023 00002000:00010000:2.0:1788845032.164612:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1187877 -> 1966505 00002000:00010000:2.0:1788845032.164652:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845031 00000100:00100000:2.0:1788845032.164689:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660267392:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.164752:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad121 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a893d expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.164786:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad121 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a893d expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.164807:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845032.164953:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641568512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10350us (11042us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.164963:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 690 00000100:00100000:1.0:1788845032.172574:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.172622:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.172701:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641568768 00000100:00100000:0.0:1788845032.172714:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641568768 00000100:00100000:0.0:1788845032.172802:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 691 00000100:00100000:0.0:1788845032.172813:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641568768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.172847:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.172922:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d0eb0cb/0xb2187301287ad128 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a894b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.173016:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660267520:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.173119:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.173351:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641568640 00000100:00100000:2.0:1788845032.173357:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641568640 00000100:00100000:2.0:1788845032.173415:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1024 00000100:00100000:2.0:1788845032.173424:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641568640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.173438:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.173456:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ad12f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8944 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.173493:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.173522:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660267648:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.173553:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845032.178085:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660267520:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845032.178172:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002d0eb0cb/0xb2187301287ad128 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a894b expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.178234:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d0eb0cb/0xb2187301287ad128 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a894b expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.178251:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030e00, rc 0) 00000100:00100000:0.0:1788845032.178401:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641568768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5586us (5774us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.178410:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 691 00000100:00100000:2.0:1788845032.179142:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660267648:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845032.179179:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:2.0:1788845032.179201:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ad12f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8944 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.179220:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.179224:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845032.179230:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ad12f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8944 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.179249:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845032.181879:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641568640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8430us (9294us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.181908:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1024 00000100:00100000:1.0:1788845032.188288:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.188339:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.188426:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641568896 00000100:00100000:0.0:1788845032.188440:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641568896 00000100:00100000:0.0:1788845032.188523:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1025 00000100:00100000:0.0:1788845032.188540:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641568896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.188569:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.188597:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad136 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8952 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.188642:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.188692:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660267776:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.188810:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.189993:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641569024 00000100:00100000:3.0:1788845032.190005:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641569024 00000100:00100000:3.0:1788845032.190065:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 692 00000100:00100000:3.0:1788845032.190074:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641569024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.190094:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.190114:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad13d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8959 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.190178:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660267904:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.190278:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.194984:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660267904:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845032.195045:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad13d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8959 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.195078:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad13d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8959 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.195115:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845032.195193:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641569024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5118us (6851us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.195205:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 692 00000100:00100000:0.0:1788845032.195560:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660267776:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845032.195581:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:0.0:1788845032.195607:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad136 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8952 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.195624:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.195627:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845032.195632:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad136 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8952 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.195646:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:0.0:1788845032.195675:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641568896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7144us (7402us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.195682:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1025 00000100:00100000:1.0:1788845032.202006:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.202061:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.202164:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641569280 00000100:00100000:2.0:1788845032.202177:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641569280 00000100:00100000:2.0:1788845032.202245:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 693 00000100:00100000:2.0:1788845032.202268:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641569280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.202295:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.202321:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad144 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8967 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.202415:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660268032:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.202524:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.202832:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641569152 00000100:00100000:3.0:1788845032.202838:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641569152 00000100:00100000:3.0:1788845032.202914:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1026 00000100:00100000:3.0:1788845032.202924:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641569152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.202940:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.202958:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad14b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8960 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.202996:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.203023:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660268160:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.203065:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.210625:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660268160:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845032.210667:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:3.0:1788845032.210694:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad14b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8960 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.210722:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.210726:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845032.210766:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad14b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8960 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.210786:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845032.210879:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641569152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7956us (8890us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.210935:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1026 00000100:00100000:2.0:1788845032.212236:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660268032:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.212298:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad144 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8967 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.212355:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad144 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8967 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.212372:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845032.212462:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641569280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10206us (10399us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.212472:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 693 00000100:00100000:1.0:1788845032.221526:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.221665:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641569408 00000100:00100000:0.0:1788845032.221687:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641569408 00000100:00100000:0.0:1788845032.221773:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1027 00000100:00100000:0.0:1788845032.221786:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641569408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.221819:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.221849:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a896e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.221924:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.221936:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.221979:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.222025:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 14 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.222041:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660268288:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845032.222140:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845032.225136:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660268288:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845032.225190:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.225217:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.225220:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845032.225225:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.225239:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:0.0:1788845032.225331:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641569408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3546us (3817us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845032.225342:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1027 00000100:00100000:1.0:1788845032.229197:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.229301:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641569536 00000100:00100000:2.0:1788845032.229315:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641569536 00010000:00010000:2.0:1788845032.229368:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 15 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.229431:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 365 00000100:00100000:2.0:1788845032.229442:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641569536:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845032.229461:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845032.229496:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 15 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.229518:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 15 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.229549:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad095 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88c6 expref: 15 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.229574:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.229605:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.229620:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.229630:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.229683:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.229701:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660268416:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845032.229818:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.233743:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.233854:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641569664 00000100:00100000:3.0:1788845032.233937:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641569664 00000100:00100000:3.0:1788845032.234003:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 281 00000100:00100000:3.0:1788845032.234016:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641569664:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845032.234213:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641569664:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 197us (494us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845032.234222:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 281 00000100:00100000:3.0:1788845032.240576:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.240671:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641569792 00000100:00100000:2.0:1788845032.240684:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641569792 00002000:00100000:2.0:1788845032.240767:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0a80 x1875739641569792/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.240781:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0a80 with x1875739641569792 00000100:00100000:2.0:1788845032.240820:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 670 00000100:00100000:2.0:1788845032.240833:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2367:x1875739641569792:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845032.243362:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660268416:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845032.243415:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845032.243494:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641569536:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 14051us (14310us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845032.243507:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 365 00000100:00100000:2.0:1788845032.261315:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2367:x1875739641569792:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 20466us (20736us total) trans 0 rc 684032/684032 00000100:00100000:2.0:1788845032.261333:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 670 00002000:00100000:2.0:1788845032.261351:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0a80 x1875739641569792/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/440 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 684032/684032 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.262375:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0a80 with x1875739641569792 00000100:00100000:3.0:1788845032.267692:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.268934:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641569920 00000100:00100000:2.0:1788845032.268952:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641569920 00000100:00100000:2.0:1788845032.269057:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 694 00000100:00100000:2.0:1788845032.269146:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641569920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.269173:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.269198:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad159 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8975 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.269276:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660268544:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.269376:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.273373:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660268544:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845032.273496:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad159 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8975 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.273528:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad159 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8975 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.273547:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845032.273656:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641569920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4514us (5988us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.273699:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 694 00000100:00100000:2.0:1788845032.281727:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.282095:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641570048 00000100:00100000:0.0:1788845032.282111:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641570048 00000100:00100000:0.0:1788845032.282189:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 695 00000100:00100000:0.0:1788845032.282202:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641570048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.282227:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.282252:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d0eb0cb/0xb2187301287ad160 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a897c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.282360:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660268672:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.282486:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.286278:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660268672:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845032.286338:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002d0eb0cb/0xb2187301287ad160 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a897c expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.286372:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d0eb0cb/0xb2187301287ad160 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a897c expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.286460:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030e00, rc 0) 00000100:00100000:1.0:1788845032.286545:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641570048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4345us (4890us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845032.286580:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 695 00000100:00100000:1.0:1788845032.295109:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.295235:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641570176 00000100:00100000:2.0:1788845032.295251:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641570176 00000100:00100000:2.0:1788845032.295545:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1028 00000100:00100000:2.0:1788845032.295559:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641570176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.295591:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.295621:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad167 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8983 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.295680:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad167 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8983 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845032.295696:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:2.0:1788845032.295707:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad167 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8983 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.295762:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.295766:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845032.295771:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845032.295867:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641570176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 311us (775us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.295875:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1028 00000100:00100000:0.0:1788845032.307863:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.308414:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641570304 00000100:00100000:2.0:1788845032.308429:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641570304 00000100:00100000:2.0:1788845032.308516:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1029 00000100:00100000:2.0:1788845032.308528:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641570304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.308558:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.308563:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845032.308634:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287ad167 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8983 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.308665:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.308683:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845032.308703:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a898a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.308746:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.308757:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.308824:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.308866:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 19 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.308940:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660268800:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845032.309225:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.319445:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660268800:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845032.319509:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.319534:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.319537:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845032.319543:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.319556:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:2.0:1788845032.319651:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641570304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11124us (11822us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.319662:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1029 00000100:00100000:1.0:1788845032.324206:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.324299:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641570432 00000100:00100000:0.0:1788845032.324312:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641570432 00010000:00010000:0.0:1788845032.324370:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 20 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.324433:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 366 00000100:00100000:0.0:1788845032.324445:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641570432:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845032.324465:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845032.324502:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 20 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.324524:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 20 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.324556:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad152 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a896e expref: 20 pid: 5840 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.324578:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.324586:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.324601:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.324634:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.324700:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.324716:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660268928:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845032.325238:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.334301:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.334353:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.334519:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641570688 00000100:00100000:3.0:1788845032.334534:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641570688 00002000:00100000:3.0:1788845032.334608:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1916927) req@ffff949f02482300 x1875739641570688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845032.334621:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02482300 with x1875739641570688 ext(1593344->1916927) 00002000:00010000:3.0:1788845032.334629:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02482300 00000100:00100000:3.0:1788845032.334672:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 376 00000100:00100000:3.0:1788845032.334684:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641570688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845032.336143:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660268928:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845032.336180:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845032.336222:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641570432:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11779us (12031us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845032.336236:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 366 00000100:00100000:2.0:1788845032.336346:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641570560 00000100:00100000:2.0:1788845032.336354:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641570560 00002000:00100000:2.0:1788845032.336418:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02483b80 x1875739641570560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.336446:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02483b80 with x1875739641570560 00000100:00100000:2.0:1788845032.336478:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 671 00000100:00100000:2.0:1788845032.336488:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641570560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845032.336754:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641570560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 267us (2474us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845032.336762:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 671 00002000:00100000:2.0:1788845032.336769:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02483b80 x1875739641570560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:138/0 lens 488/4536 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.336777:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02483b80 with x1875739641570560 00000100:00100000:3.0:1788845032.343456:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641570688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8756us (9084us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845032.343476:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 376 00002000:00100000:3.0:1788845032.343492:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1916927) req@ffff949f02482300 x1875739641570688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:138/0 lens 488/440 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845032.343504:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02482300 with x1875739641570688 ext(1593344->1916927) 00002000:00010000:3.0:1788845032.343512:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02482300 00000100:00100000:1.0:1788845032.345636:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.345755:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641570816 00000100:00100000:0.0:1788845032.345771:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641570816 00002000:00100000:0.0:1788845032.345867:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966d80 x1875739641570816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845032.345909:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966d80 with x1875739641570816 00000100:00100000:0.0:1788845032.345956:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 672 00000100:00100000:0.0:1788845032.345968:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641570816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845032.346328:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641570816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 360us (707us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845032.346337:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 672 00002000:00100000:0.0:1788845032.346345:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966d80 x1875739641570816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:138/0 lens 488/4536 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845032.346372:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966d80 with x1875739641570816 00000100:00100000:0.0:1788845032.353334:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.353506:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641570944 00000100:00100000:2.0:1788845032.353520:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641570944 00002000:00100000:2.0:1788845032.353586:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383d180 x1875739641570944/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:138/0 lens 8680/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.353600:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383d180 with x1875739641570944 00000100:00100000:2.0:1788845032.353647:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 673 00000100:00100000:2.0:1788845032.353658:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641570944:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845032.354555:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 683971 00000004:00010000:2.0:1788845032.354563:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845032 -> 1788845032 00000004:00010000:2.0:1788845032.354566:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:2.0:1788845032.354568:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845032 -> 1788845032 00000004:00010000:2.0:1788845032.354570:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:2.0:1788845032.354672:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641570944:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 1011us (1350us total) trans 4294967685 rc 0/0 00000100:00100000:2.0:1788845032.354696:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 673 00002000:00100000:2.0:1788845032.354706:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383d180 x1875739641570944/t4294967685(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:138/0 lens 8680/448 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.354714:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383d180 with x1875739641570944 00000100:00100000:1.0:1788845032.359748:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.359963:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641571072 00000100:00100000:3.0:1788845032.359978:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641571072 00000100:00100000:3.0:1788845032.360061:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1030 00000100:00100000:3.0:1788845032.360073:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2367:x1875739641571072:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:2.0:1788845032.456833:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967684 is committed 00000001:00080000:2.0:1788845032.456868:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967679 is committed 00000001:00080000:2.0:1788845032.456917:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967685 is committed 00000001:00080000:2.0:1788845032.456933:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967681 is committed 00000100:00100000:0.0:1788845032.457278:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641571072:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 97199us (97546us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845032.457292:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1030 00000100:00100000:1.0:1788845032.469805:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.470965:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641571200 00000100:00100000:0.0:1788845032.470980:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641571200 00000100:00100000:0.0:1788845032.471073:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1031 00000100:00100000:0.0:1788845032.471086:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641571200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.471115:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.471145:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad175 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8991 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.471208:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.471263:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660269056:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.471511:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.483752:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.483947:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641571328 00000100:00100000:3.0:1788845032.483961:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641571328 00000100:00100000:3.0:1788845032.484207:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 696 00000100:00100000:3.0:1788845032.484219:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641571328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.484265:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.484291:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ad17c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8998 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.484376:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660269184:192.168.201.29@tcp:106: 00000004:00010000:0.0:1788845032.484399:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845031 -> 1788845032 00000100:00100000:0.0:1788845032.484429:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660269056:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845032.484458:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:0.0:1788845032.484479:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad175 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8991 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.484485:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00010000:00010000:0.0:1788845032.484500:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.484510:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845032.484516:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad175 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8991 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.484534:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:0.0:1788845032.484568:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641571200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13486us (14779us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.484578:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1031 00002000:00010000:3.0:1788845032.487304:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845031 -> 1788845032 00000100:00100000:3.0:1788845032.487354:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660269184:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845032.487407:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ad17c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8998 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.487440:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ad17c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8998 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.487453:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845032.487537:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641571328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3319us (3799us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.487548:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 696 00000100:00100000:1.0:1788845032.499136:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.499309:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.499994:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641571456 00000100:00100000:3.0:1788845032.500011:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641571456 00000100:00100000:3.0:1788845032.500241:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1032 00000100:00100000:3.0:1788845032.500255:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641571456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.500286:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.500313:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad183 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a899f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.500390:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.500451:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660269312:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.500582:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.510494:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660269312:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845032.510536:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:3.0:1788845032.510565:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad183 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a899f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.510591:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.510594:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845032.510600:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad183 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a899f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.510626:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845032.510712:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641571456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10459us (11401us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.510721:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1032 00000100:00100000:2.0:1788845032.511000:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641571584 00000100:00100000:2.0:1788845032.511012:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641571584 00000100:00100000:2.0:1788845032.511071:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 697 00000100:00100000:2.0:1788845032.511083:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641571584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.511104:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.511128:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad18a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89a6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.511344:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660269440:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.511494:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.516711:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660269440:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.516938:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad18a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89a6 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.516974:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad18a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89a6 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.517019:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845032.518466:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641571584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7368us (19331us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.518482:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 697 00000100:00100000:0.0:1788845032.529605:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.529666:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.529837:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641571712 00000100:00100000:2.0:1788845032.529859:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641571712 00000100:00100000:2.0:1788845032.529967:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1033 00000100:00100000:2.0:1788845032.529979:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641571712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.530008:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.530053:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ad191 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89ad expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.530111:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.530170:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660269568:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.530276:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.530552:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641571840 00000100:00100000:3.0:1788845032.530558:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641571840 00000100:00100000:3.0:1788845032.530608:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 698 00000100:00100000:3.0:1788845032.530617:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641571840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.530629:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.530643:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ad198 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89b4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.530707:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660269696:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.530751:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.536419:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660269696:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845032.536485:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ad198 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89b4 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.536518:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ad198 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89b4 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.536534:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845032.536622:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641571840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6005us (6953us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.536632:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 698 00000100:00100000:2.0:1788845032.537450:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660269568:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845032.537474:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:2.0:1788845032.537492:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ad191 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ad expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.537511:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.537515:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845032.537542:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287ad191 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ad expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.537558:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845032.537600:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641571712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7624us (8011us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.537608:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1033 00000100:00100000:1.0:1788845032.545341:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.545400:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.545496:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641572096 00000100:00100000:2.0:1788845032.545510:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641572096 00000100:00100000:2.0:1788845032.545589:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 699 00000100:00100000:2.0:1788845032.545601:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641572096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.545629:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.545655:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad19f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89c2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.545737:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660269824:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.545840:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845032.546079:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641571968 00000100:00100000:0.0:1788845032.546085:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641571968 00000100:00100000:0.0:1788845032.546133:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1034 00000100:00100000:0.0:1788845032.546142:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641571968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.546155:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.546173:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1a6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89bb expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.546214:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.546246:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660269952:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.546283:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.549783:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660269824:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.549861:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad19f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89c2 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.549940:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad19f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89c2 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.549959:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845032.551069:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641572096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4433us (4631us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.551109:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 699 00000100:00100000:3.0:1788845032.555702:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660269952:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845032.555754:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:3.0:1788845032.555784:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1a6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89bb expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.555812:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.555815:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845032.555821:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89bb expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.555859:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:3.0:1788845032.555991:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641571968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9849us (10665us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.556000:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1034 00000100:00100000:1.0:1788845032.570092:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.570203:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641572224 00000100:00100000:0.0:1788845032.570220:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641572224 00000100:00100000:0.0:1788845032.570311:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1035 00000100:00100000:0.0:1788845032.570325:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641572224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.570389:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.570416:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad1ad lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89c9 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.570471:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.570531:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660270080:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.570653:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.571499:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.571576:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641572352 00000100:00100000:2.0:1788845032.571586:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641572352 00000100:00100000:2.0:1788845032.571639:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 700 00000100:00100000:2.0:1788845032.571650:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641572352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.571668:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.571687:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad1b4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89d0 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.571742:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660270208:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.571809:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.575679:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660270208:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.575748:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad1b4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d0 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.575779:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad1b4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d0 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.575798:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845032.575938:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641572352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4233us (4387us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.575950:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 700 00000100:00100000:0.0:1788845032.576089:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660270080:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845032.576110:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:0.0:1788845032.576124:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad1ad lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89c9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.576140:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.576143:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845032.576148:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad1ad lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89c9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.576161:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:0.0:1788845032.576189:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641572224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5867us (6113us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.576197:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1035 00000100:00100000:1.0:1788845032.582327:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.582482:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641572480 00000100:00100000:3.0:1788845032.582496:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641572480 00000100:00100000:3.0:1788845032.582611:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1036 00000100:00100000:3.0:1788845032.582624:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641572480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.582653:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.582681:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89d7 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.582721:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.582742:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.582786:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.582807:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 10 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.582823:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660270336:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845032.582963:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.586168:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660270336:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845032.586224:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.586271:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.586274:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845032.586280:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.586292:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845032.586377:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641572480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3748us (4059us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845032.586388:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1036 00000100:00100000:1.0:1788845032.591268:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.591361:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641572608 00000100:00100000:3.0:1788845032.591383:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641572608 00010000:00010000:3.0:1788845032.591437:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 11 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.591479:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 367 00000100:00100000:3.0:1788845032.591488:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641572608:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845032.591501:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845032.591532:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 11 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.591549:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 11 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.591572:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287ad16e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a898a expref: 11 pid: 11036 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.591612:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.591618:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.591629:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.591639:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.591688:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.591706:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660270464:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845032.591835:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.599277:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660270464:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845032.599318:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845032.599406:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641572608:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7916us (8155us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845032.599419:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 367 00000100:00100000:0.0:1788845032.601075:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.601186:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641572736 00000100:00100000:2.0:1788845032.601197:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641572736 00002000:00100000:2.0:1788845032.601277:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383c700 x1875739641572736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.601289:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c700 with x1875739641572736 00000100:00100000:2.0:1788845032.601335:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 674 00000100:00100000:2.0:1788845032.601346:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641572736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845032.601638:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641572736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 292us (573us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845032.601647:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 674 00002000:00100000:2.0:1788845032.601653:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383c700 x1875739641572736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/4536 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.601660:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c700 with x1875739641572736 00000100:00100000:1.0:1788845032.609570:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.609991:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641572864 00000100:00100000:3.0:1788845032.610005:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641572864 00002000:00100000:3.0:1788845032.610075:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009687100 x1875739641572864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845032.610087:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009687100 with x1875739641572864 00000100:00100000:3.0:1788845032.610158:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 675 00000100:00100000:3.0:1788845032.610170:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641572864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845032.610519:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641572864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 349us (964us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845032.610528:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 675 00002000:00100000:3.0:1788845032.610536:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009687100 x1875739641572864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/4536 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845032.610545:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009687100 with x1875739641572864 00000100:00100000:3.0:1788845032.617287:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.619782:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641572992 00000100:00100000:2.0:1788845032.619808:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641572992 00002000:00100000:2.0:1788845032.619938:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1180 x1875739641572992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.619953:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1180 with x1875739641572992 00000100:00100000:2.0:1788845032.620005:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 676 00000100:00100000:2.0:1788845032.620017:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641572992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845032.620346:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641572992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 331us (3075us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845032.620354:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 676 00002000:00100000:2.0:1788845032.620362:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1180 x1875739641572992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/4536 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.620369:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1180 with x1875739641572992 00000100:00100000:0.0:1788845032.628475:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.628678:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641573120 00000100:00100000:3.0:1788845032.628692:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641573120 00002000:00100000:3.0:1788845032.628776:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383dc00 x1875739641573120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845032.628790:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383dc00 with x1875739641573120 00000100:00100000:3.0:1788845032.628838:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 677 00000100:00100000:3.0:1788845032.628860:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641573120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845032.629400:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641573120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 548us (939us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845032.629409:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 677 00002000:00100000:3.0:1788845032.629418:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383dc00 x1875739641573120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/4536 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845032.629425:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383dc00 with x1875739641573120 00000100:00100000:0.0:1788845032.640549:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.640738:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641573248 00000100:00100000:2.0:1788845032.640752:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641573248 00002000:00100000:2.0:1788845032.640825:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383f800 x1875739641573248/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 16872/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.640838:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383f800 with x1875739641573248 00000100:00100000:2.0:1788845032.640919:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 678 00000100:00100000:2.0:1788845032.640931:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641573248:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845032.641516:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 683971 00000004:00010000:2.0:1788845032.641521:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845032 -> 1788845032 00000004:00010000:2.0:1788845032.641524:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845032 -> 1788845031 00000004:00010000:2.0:1788845032.641526:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845032 -> 1788845032 00000004:00010000:2.0:1788845032.641529:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:2.0:1788845032.644084:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641573248:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3135us (3531us total) trans 4294967686 rc 0/0 00000100:00100000:2.0:1788845032.644102:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 678 00002000:00100000:2.0:1788845032.644121:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383f800 x1875739641573248/t4294967686(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 16872/448 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845032.644133:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383f800 with x1875739641573248 00000100:00100000:0.0:1788845032.648231:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.648447:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641573376 00000100:00100000:2.0:1788845032.648461:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641573376 00000100:00100000:2.0:1788845032.648544:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1037 00000100:00100000:2.0:1788845032.648557:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641573376:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845032.704038:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967686 is committed 00000100:00100000:1.0:1788845032.704296:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641573376:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 55733us (56077us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845032.704312:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1037 00000100:00100000:1.0:1788845032.712608:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.712938:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641573504 00000100:00100000:2.0:1788845032.712964:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641573504 00000100:00100000:2.0:1788845032.713052:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 701 00000100:00100000:2.0:1788845032.713089:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641573504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.713117:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.713143:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad1c2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89de expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.713225:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660270592:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.713354:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.721490:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660270592:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.721673:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad1c2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89de expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.721705:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287ad1c2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89de expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.721722:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:2.0:1788845032.721821:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641573504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8735us (9228us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.721832:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 701 00000100:00100000:1.0:1788845032.732949:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.733510:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641573632 00000100:00100000:3.0:1788845032.733554:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641573632 00000100:00100000:3.0:1788845032.733637:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 702 00000100:00100000:3.0:1788845032.733649:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641573632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.733675:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.733700:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad1c9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89e5 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.733790:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660270720:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.733946:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.742252:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660270720:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845032.742341:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad1c9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89e5 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.742630:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad1c9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89e5 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.742648:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:3.0:1788845032.742779:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641573632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9130us (9896us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.742789:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 702 00000100:00100000:1.0:1788845032.749085:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.749196:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641573760 00000100:00100000:0.0:1788845032.749210:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641573760 00000100:00100000:0.0:1788845032.749455:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1038 00000100:00100000:0.0:1788845032.749471:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641573760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.749501:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.749608:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89ec expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.749985:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.749999:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.750046:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.750070:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 14 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.750089:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660270848:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845032.750265:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.760087:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.760236:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660270848:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845032.760295:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.760324:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.760347:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845032.760353:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.760377:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:0.0:1788845032.760461:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641573760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10994us (11391us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845032.760472:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1038 00000100:00100000:2.0:1788845032.763721:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641573888 00000100:00100000:2.0:1788845032.764011:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641573888 00010000:00010000:2.0:1788845032.764083:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 15 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.764135:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 368 00000100:00100000:2.0:1788845032.764148:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641573888:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845032.764174:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845032.764213:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 15 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.764236:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 15 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.764266:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad1bb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89d7 expref: 15 pid: 5841 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.764288:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.764294:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.764307:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.764317:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.764602:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.764624:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660270976:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845032.764811:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.774352:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.774755:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660270976:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845032.774797:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845032.774914:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641573888:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10740us (14813us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.774928:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 368 00000100:00100000:3.0:1788845032.777510:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641574016 00000100:00100000:3.0:1788845032.777603:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641574016 00000100:00100000:2.0:1788845032.777957:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 679 00000100:00100000:2.0:1788845032.777972:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641574016:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845032.778450:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 683971 00000004:00010000:2.0:1788845032.778456:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845032 -> 1788845032 00000004:00010000:2.0:1788845032.778459:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:2.0:1788845032.778462:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845032 -> 1788845032 00000004:00010000:2.0:1788845032.778465:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:2.0:1788845032.778757:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641574016:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 785us (4421us total) trans 4294967687 rc 0/0 00000100:00100000:2.0:1788845032.778768:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 679 00000100:00100000:1.0:1788845032.783543:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.786814:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641574272 00000100:00100000:0.0:1788845032.786837:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641574272 00000100:00100000:0.0:1788845032.786983:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1039 00000100:00100000:0.0:1788845032.786997:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641574272:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845032.787082:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845032.787120:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287ad105 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8928 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.787148:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.787154:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845032.787470:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.787489:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.787500:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.788055:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.788108:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 13 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.788129:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660271104:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845032.788286:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845032.793109:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660271104:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845032.793160:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845032.801394:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.801977:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641574400 00000100:00100000:2.0:1788845032.801992:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641574400 00010000:00010000:2.0:1788845032.802069:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 14 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.802123:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 369 00000100:00100000:2.0:1788845032.802136:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d4bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641574400:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845032.802155:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845032.802164:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 14 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.802190:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 14 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.802217:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287ad113 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a892f expref: 14 pid: 5839 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.802239:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.802244:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.802266:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.802274:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.802308:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845032.802381:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d4bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641574400:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 247us (1010us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.802389:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 369 00010000:00010000:0.0:1788845032.803007:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.803018:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845032.803194:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967688 00000004:00010000:0.0:1788845032.803278:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 683971 00000004:00010000:0.0:1788845032.803283:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845032 -> 1788845032 00000004:00010000:0.0:1788845032.803285:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:0.0:1788845032.803288:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845032 -> 1788845032 00000004:00010000:0.0:1788845032.803290:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:0.0:1788845032.803300:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e6432906 reply state 0000000045ebacda transno 4294967689 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.803309:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845032.803426:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641574272:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 16428us (19907us total) trans 4294967689 rc 0/0 00000100:00100000:0.0:1788845032.803455:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1039 00000020:00080000:0.0:1788845032.803461:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845032.807053:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.807079:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.807108:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.807115:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a000031800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845032.807121:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.807130:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.807167:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.807170:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845032.807177:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.807184:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1d7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845032.824337:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.824424:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641574528 00000100:00100000:0.0:1788845032.824436:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641574528 00002000:00010000:0.0:1788845032.824486:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1421197->18446744073709551615) 00002000:00100000:0.0:1788845032.824490:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009686a00 with x1875739641574528 ext(1421197->18446744073709551615) 00002000:00010000:0.0:1788845032.824501:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009686a00. 00000100:00100000:0.0:1788845032.824534:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 377 00000100:00100000:0.0:1788845032.824544:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641574528:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845032.824781:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1966505 -> 1421197 00002000:00010000:0.0:1788845032.824785:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845032 -> 1788845032 00002000:00010000:0.0:1788845032.824788:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845032 -> 1788845029 00002000:00010000:0.0:1788845032.824790:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845032 -> 1788845032 00002000:00010000:0.0:1788845032.824792:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 15 -> 15 00000100:00100000:0.0:1788845032.824866:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641574528:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 321us (538us total) trans 4294967424 rc 0/0 00000100:00100000:0.0:1788845032.824874:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 377 00002000:00010000:0.0:1788845032.824878:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1421197->18446744073709551615) 00002000:00100000:0.0:1788845032.824881:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009686a00 with x1875739641574528 ext(1421197->18446744073709551615) 00002000:00010000:0.0:1788845032.824920:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009686a00. 00000100:00100000:1.0:1788845032.829358:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.829487:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641574656 00000100:00100000:3.0:1788845032.829505:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641574656 00000100:00100000:3.0:1788845032.829594:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1040 00000100:00100000:3.0:1788845032.829607:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641574656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.829635:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.829663:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287ad1de lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89f3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.829779:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad1e5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.829798:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad1e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.829832:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad1e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845032.829876:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1421197 00000004:00200000:3.0:1788845032.829930:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845032.829958:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad1e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.829994:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287ad1de lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89f3 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.830014:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad1e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89f3 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.830034:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.830038:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845032.830041:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e200, rc 0) 00000100:00100000:3.0:1788845032.830129:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641574656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 524us (780us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845032.830137:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1040 00000100:00100000:2.0:1788845032.840417:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.840466:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.840633:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641574912 00000100:00100000:0.0:1788845032.840646:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641574912 00000100:00100000:0.0:1788845032.840720:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 703 00000100:00100000:0.0:1788845032.840733:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641574912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.840761:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.840799:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287ad1ec lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a01 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.840921:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660271232:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.841038:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845032.841388:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641574784 00000100:00100000:1.0:1788845032.841394:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641574784 00000100:00100000:1.0:1788845032.841447:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1041 00000100:00100000:1.0:1788845032.841456:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641574784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845032.841471:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845032.841489:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ad1f3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a89fa expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.841540:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845032.841574:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beabb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660271360:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845032.841619:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f0c0 going to sleep for 11 seconds 00002000:00010000:0.0:1788845032.858433:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845031 00000100:00100000:0.0:1788845032.858475:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660271232:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845032.858533:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287ad1ec lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a01 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.858562:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287ad1ec lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a01 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.858579:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:0.0:1788845032.858657:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641574912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17925us (18188us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.858667:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 703 00000100:00100000:1.0:1788845032.858935:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beabb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660271360:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845032.858954:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:1.0:1788845032.858969:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ad1f3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89fa expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.858984:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845032.858987:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845032.858992:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ad1f3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89fa expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.859003:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b800, rc 0) 00000100:00100000:1.0:1788845032.859065:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641574784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17611us (18664us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845032.859074:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1041 00000100:00100000:1.0:1788845032.865551:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.865664:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641575040 00000100:00100000:0.0:1788845032.865678:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641575040 00000100:00100000:0.0:1788845032.865754:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1042 00000100:00100000:0.0:1788845032.865766:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641575040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.865797:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.865823:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1fa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a08 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.865914:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.865990:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660271488:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.866114:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.872064:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.872113:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641575168 00000100:00100000:3.0:1788845032.872132:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641575168 00000100:00100000:3.0:1788845032.872213:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 704 00000100:00100000:3.0:1788845032.872225:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641575168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.872245:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.872265:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad201 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a0f expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.872353:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660271616:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.872392:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660271488:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845032.872431:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:0.0:1788845032.872456:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1fa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a08 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.872465:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00010000:00010000:0.0:1788845032.872485:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.872492:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845032.872499:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287ad1fa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a08 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.872533:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:0.0:1788845032.872595:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641575040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6829us (7058us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.872605:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1042 00000100:00100000:3.0:1788845032.881461:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660271616:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845032.881504:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad201 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a0f expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.881529:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad201 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a0f expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.881543:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:3.0:1788845032.881616:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641575168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9393us (9578us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.881625:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 704 00000100:00100000:1.0:1788845032.887080:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.887217:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641575296 00000100:00100000:0.0:1788845032.887235:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641575296 00000100:00100000:0.0:1788845032.887316:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1043 00000100:00100000:0.0:1788845032.887330:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641575296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.887358:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1788845032.887380:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845032.887394:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad208 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a16 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.887442:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.887513:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660271744:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845032.887627:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.887855:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641575424 00000100:00100000:3.0:1788845032.887861:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641575424 00000100:00100000:3.0:1788845032.887935:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 705 00000100:00100000:3.0:1788845032.887943:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641575424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.887954:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.887969:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad20f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a1d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.888013:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660271872:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.888064:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845032.893658:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660271744:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845032.893689:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:0.0:1788845032.893709:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad208 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a16 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.893730:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.893734:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845032.893740:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287ad208 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a16 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.893756:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:0.0:1788845032.894281:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641575296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6952us (7211us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845032.894291:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1043 00000100:00100000:3.0:1788845032.904311:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660271872:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845032.904381:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad20f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a1d expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.904415:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad20f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a1d expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.904434:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845032.904605:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641575424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16661us (17226us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.904616:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 705 00000100:00100000:1.0:1788845032.918142:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.918201:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845032.918308:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641575680 00000100:00100000:2.0:1788845032.918317:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641575680 00000100:00100000:2.0:1788845032.918385:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 706 00000100:00100000:2.0:1788845032.918395:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641575680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845032.918414:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845032.918431:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad216 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a2b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845032.918491:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660272000:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845032.918577:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845032.919094:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641575552 00000100:00100000:3.0:1788845032.919100:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641575552 00000100:00100000:3.0:1788845032.919182:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1044 00000100:00100000:3.0:1788845032.919191:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641575552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845032.919204:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845032.919219:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad21d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a24 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.919315:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845032.919344:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660272128:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845032.919396:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.927766:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660272000:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845032.927817:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad216 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a2b expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.927857:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad216 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a2b expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.927872:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:2.0:1788845032.928204:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641575680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9808us (10000us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845032.928213:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 706 00000100:00100000:3.0:1788845032.928322:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660272128:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845032.928354:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:3.0:1788845032.928379:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad21d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a24 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.928394:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845032.928397:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845032.928402:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad21d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a24 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845032.928413:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845032.928442:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641575552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9253us (10311us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845032.928449:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1044 00000100:00100000:2.0:1788845032.937657:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845032.937913:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641575808 00000100:00100000:1.0:1788845032.937923:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641575808 00000100:00100000:1.0:1788845032.937984:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1045 00000100:00100000:1.0:1788845032.937995:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641575808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845032.938018:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845032.938040:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a32 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.938069:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.938078:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.938112:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845032.938131:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 12 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845032.938146:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660272256:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845032.938337:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.943904:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660272256:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845032.943947:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.943968:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845032.943972:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845032.943977:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845032.943989:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ac00, rc 0) 00000100:00100000:2.0:1788845032.944133:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641575808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6139us (6484us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.944143:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1045 00000100:00100000:3.0:1788845032.946159:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.946230:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641575936 00000100:00100000:0.0:1788845032.946242:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641575936 00010000:00010000:0.0:1788845032.946301:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 13 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.946347:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 370 00000100:00100000:0.0:1788845032.946358:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641575936:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845032.946382:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845032.946409:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 13 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.946441:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 13 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.946464:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287ad1d0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a89ec expref: 13 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.946480:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.946486:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.946497:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.946507:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.946545:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845032.946558:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660272384:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845032.947185:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845032.949526:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660272384:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845032.949554:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845032.949600:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641575936:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3244us (3450us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845032.949611:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 370 00000100:00100000:2.0:1788845032.957554:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845032.957643:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641576064 00000100:00100000:3.0:1788845032.957653:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641576064 00002000:00100000:3.0:1788845032.957700:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656a00 x1875739641576064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/0 e 0 to 0 dl 1788845043 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845032.957712:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656a00 with x1875739641576064 00000100:00100000:2.0:1788845032.958083:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 680 00000100:00100000:2.0:1788845032.958096:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641576064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845032.988708:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641576064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 30605us (31156us total) trans 0 rc 684032/684032 00000100:00100000:0.0:1788845032.991664:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 680 00002000:00100000:0.0:1788845032.991687:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656a00 x1875739641576064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:138/0 lens 488/440 e 0 to 0 dl 1788845043 ref 1 fl Complete:/600/0 rc 684032/684032 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845032.991726:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656a00 with x1875739641576064 00000100:00100000:2.0:1788845032.994541:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845032.994642:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641576192 00000100:00100000:0.0:1788845032.994653:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641576192 00000100:00100000:0.0:1788845032.994715:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1046 00000100:00100000:0.0:1788845032.997280:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641576192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845032.997310:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845032.997334:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ad22b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a39 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.997437:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.997452:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.997463:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845032.997491:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1421197 00000004:00200000:0.0:1788845032.997496:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845032.997514:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.997540:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287ad22b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.997558:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845032.997587:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845032.997590:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845032.997593:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864400, rc 0) 00000100:00100000:0.0:1788845032.997655:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641576192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 396us (3123us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.001075:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1046 00000100:00100000:1.0:1788845033.001240:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.001325:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641576320 00000100:00100000:0.0:1788845033.001335:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641576320 00000100:00100000:0.0:1788845033.001490:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1047 00000100:00100000:0.0:1788845033.001502:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641576320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.001524:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.001544:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ad239 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a40 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.001579:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ad239 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a40 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845033.001592:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 683971 00010000:00010000:0.0:1788845033.001599:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ad239 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a40 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.001612:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.001615:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.001619:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:0.0:1788845033.001679:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641576320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 180us (439us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.001701:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1047 00000100:00100000:3.0:1788845033.004598:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.004681:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641576448 00000100:00100000:0.0:1788845033.004692:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641576448 00000100:00080000:1.0:1788845033.004837:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788845033.004842:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845033.005640:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 707 00000100:00100000:1.0:1788845033.005653:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641576448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.005671:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.005690:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a463c78/0xb2187301287ad240 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a47 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.005759:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beaa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660272512:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845033.005854:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.007986:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beaa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660272512:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845033.008026:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a463c78/0xb2187301287ad240 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a47 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.008050:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a463c78/0xb2187301287ad240 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a47 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.008063:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b200, rc 0) 00000100:00100000:1.0:1788845033.008123:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641576448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2473us (3534us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845033.008134:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 707 00000100:00100000:0.0:1788845033.011994:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.012085:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641576576 00000100:00100000:3.0:1788845033.012095:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641576576 00000100:00100000:3.0:1788845033.012155:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 708 00000100:00100000:3.0:1788845033.012165:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641576576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.012184:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.012202:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad247 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a4e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.012285:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660272640:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.012383:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.020706:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660272640:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845033.020764:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad247 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a4e expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.020790:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad247 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a4e expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.020804:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:0.0:1788845033.020924:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641576576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8716us (8897us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.020934:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 708 00000100:00100000:0.0:1788845033.024736:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.024817:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641576704 00000100:00100000:3.0:1788845033.024826:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641576704 00000100:00100000:3.0:1788845033.025140:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 709 00000100:00100000:3.0:1788845033.025150:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641576704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.025170:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.025186:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad24e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a55 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.025246:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660272768:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.025331:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.028601:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660272768:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.028645:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad24e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a55 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.028670:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad24e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a55 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.028683:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845033.028754:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641576704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3605us (4041us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.028763:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 709 00000100:00100000:2.0:1788845033.033959:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.034185:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641576832 00000100:00100000:3.0:1788845033.034195:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641576832 00000100:00100000:3.0:1788845033.034252:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 710 00000100:00100000:3.0:1788845033.034271:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641576832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.034290:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.034307:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad255 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a5c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.034376:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660272896:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.034463:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.037834:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660272896:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.037923:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad255 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a5c expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.037948:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad255 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a5c expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.037963:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845033.038022:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641576832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3752us (4072us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.038030:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 710 00000100:00100000:1.0:1788845033.040783:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.041109:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641576960 00000100:00100000:3.0:1788845033.041118:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641576960 00002000:00100000:3.0:1788845033.041173:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beaa300 x1875739641576960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.041184:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beaa300 with x1875739641576960 00000100:00100000:3.0:1788845033.041239:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 681 00000100:00100000:3.0:1788845033.041248:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641576960:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.041508:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641576960:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (732us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.041515:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 681 00002000:00100000:3.0:1788845033.041521:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beaa300 x1875739641576960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.041529:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beaa300 with x1875739641576960 00000100:00080000:1.0:1788845033.041655:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788845033.042684:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845033.045430:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.045538:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641577088 00000100:00100000:1.0:1788845033.045547:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641577088 00000100:00100000:1.0:1788845033.045608:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 711 00000100:00100000:1.0:1788845033.045618:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641577088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.045636:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.045653:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287ad25c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a63 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.045711:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660273024:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845033.045806:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.051380:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660273024:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845033.051423:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287ad25c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a63 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.051450:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007fabafa7/0xb2187301287ad25c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a63 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.051463:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1ba00, rc 0) 00000100:00100000:0.0:1788845033.051544:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641577088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5927us (6122us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.051554:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 711 00000100:00100000:2.0:1788845033.055391:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.055483:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641577216 00000100:00100000:1.0:1788845033.055492:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641577216 00002000:00100000:1.0:1788845033.055539:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656300 x1875739641577216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.055550:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656300 with x1875739641577216 00000100:00100000:1.0:1788845033.055590:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 682 00000100:00100000:1.0:1788845033.055600:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641577216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845033.056100:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641577216:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 499us (717us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.056108:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 682 00002000:00100000:1.0:1788845033.056116:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656300 x1875739641577216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.056125:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656300 with x1875739641577216 00000100:00100000:2.0:1788845033.060746:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.060827:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641577344 00000100:00100000:0.0:1788845033.060838:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641577344 00000100:00100000:0.0:1788845033.060945:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 712 00000100:00100000:0.0:1788845033.060956:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641577344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.060975:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.060994:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ffc4f556/0xb2187301287ad263 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a6a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.061050:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660273152:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.061129:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.068448:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660273152:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.068492:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ffc4f556/0xb2187301287ad263 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a6a expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.068517:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffc4f556/0xb2187301287ad263 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a6a expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.068530:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865e00, rc 0) 00000100:00100000:3.0:1788845033.068594:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641577344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7640us (7856us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.068603:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 712 00000100:00100000:2.0:1788845033.078216:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.078313:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641577472 00000100:00100000:3.0:1788845033.078323:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641577472 00002000:00100000:3.0:1788845033.078382:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654380 x1875739641577472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.078394:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654380 with x1875739641577472 00000100:00100000:3.0:1788845033.078444:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 683 00000100:00100000:3.0:1788845033.078454:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641577472:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.078704:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641577472:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (496us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.078712:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 683 00002000:00100000:3.0:1788845033.078729:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654380 x1875739641577472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.078737:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654380 with x1875739641577472 00000100:00100000:2.0:1788845033.093402:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.093560:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641577600 00000100:00100000:3.0:1788845033.093569:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641577600 00000100:00100000:3.0:1788845033.093629:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 713 00000100:00100000:3.0:1788845033.093640:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641577600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.093660:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.093678:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad26a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a71 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.093748:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660273280:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.093855:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.096508:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660273280:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.096623:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad26a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a71 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.096648:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f36a4a29/0xb2187301287ad26a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a71 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.096679:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845033.096806:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641577600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3166us (3412us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.096815:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 713 00000100:00100000:2.0:1788845033.100775:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.100981:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641577728 00000100:00100000:1.0:1788845033.100991:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641577728 00002000:00100000:1.0:1788845033.101053:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654e00 x1875739641577728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.101064:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654e00 with x1875739641577728 00000100:00100000:1.0:1788845033.101104:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 684 00000100:00100000:1.0:1788845033.101114:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641577728:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845033.101358:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641577728:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 244us (591us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.102190:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 684 00002000:00100000:1.0:1788845033.102202:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654e00 x1875739641577728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.102211:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654e00 with x1875739641577728 00000100:00100000:2.0:1788845033.105996:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.106131:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641577856 00000100:00100000:3.0:1788845033.106142:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641577856 00000100:00100000:3.0:1788845033.106193:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 714 00000100:00100000:3.0:1788845033.106203:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641577856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.106222:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.106240:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad271 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a78 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.106297:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660273408:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.106391:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.108858:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660273408:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845033.108923:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad271 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a78 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.108949:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad271 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a78 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.108973:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:2.0:1788845033.109025:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641577856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2823us (3038us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.109035:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 714 00000100:00100000:2.0:1788845033.114991:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.115175:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641577984 00000100:00100000:3.0:1788845033.115185:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641577984 00002000:00100000:3.0:1788845033.115233:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5180 x1875739641577984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.115244:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5180 with x1875739641577984 00000100:00100000:3.0:1788845033.115280:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 685 00000100:00100000:3.0:1788845033.115291:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641577984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.116103:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641577984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 810us (1116us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.116112:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 685 00002000:00100000:3.0:1788845033.116134:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5180 x1875739641577984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.116142:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5180 with x1875739641577984 00000100:00100000:2.0:1788845033.120621:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.120773:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641578112 00000100:00100000:0.0:1788845033.120783:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641578112 00000100:00100000:0.0:1788845033.120855:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 715 00000100:00100000:0.0:1788845033.120865:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641578112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.121069:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.121088:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287ad278 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a7f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.121148:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660273536:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.121230:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.124197:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660273536:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845033.124238:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287ad278 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a7f expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.124281:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287ad278 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a7f expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.124295:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:0.0:1788845033.124413:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641578112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3549us (3800us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.124422:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 715 00000100:00100000:2.0:1788845033.128643:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.128746:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641578240 00000100:00100000:1.0:1788845033.128757:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641578240 00002000:00100000:1.0:1788845033.128809:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc7480 x1875739641578240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.128821:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc7480 with x1875739641578240 00000100:00100000:1.0:1788845033.128860:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 686 00000100:00100000:1.0:1788845033.128871:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641578240:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845033.129335:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641578240:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 463us (699us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.129342:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 686 00002000:00100000:1.0:1788845033.129350:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc7480 x1875739641578240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.129357:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc7480 with x1875739641578240 00000100:00100000:2.0:1788845033.133830:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.134066:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641578368 00000100:00100000:3.0:1788845033.134076:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641578368 00000100:00100000:3.0:1788845033.134135:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 716 00000100:00100000:3.0:1788845033.134145:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641578368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.134164:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.134182:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287ad27f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a86 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.134241:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660273664:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.134325:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.138616:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660273664:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845033.138659:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287ad27f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a86 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.138684:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287ad27f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a86 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.138697:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f400, rc 0) 00000100:00100000:2.0:1788845033.138771:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641578368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4626us (4951us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.138780:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 716 00000100:00100000:2.0:1788845033.149198:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.149319:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641578496 00000100:00100000:0.0:1788845033.149328:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641578496 00002000:00100000:0.0:1788845033.149388:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6d80 x1875739641578496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.149399:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6d80 with x1875739641578496 00000100:00100000:0.0:1788845033.149434:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 687 00000100:00100000:0.0:1788845033.149444:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641578496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.149718:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641578496:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (531us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.149726:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 687 00002000:00100000:0.0:1788845033.149745:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6d80 x1875739641578496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.149753:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6d80 with x1875739641578496 00000100:00100000:3.0:1788845033.154158:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.154335:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641578624 00000100:00100000:2.0:1788845033.154345:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641578624 00000100:00100000:2.0:1788845033.154414:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 717 00000100:00100000:2.0:1788845033.154423:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641578624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845033.154444:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845033.154464:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad286 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a8d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.154518:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660273792:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845033.154612:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.158222:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660273792:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845033.158266:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad286 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a8d expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.158291:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad286 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a8d expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.158304:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:2.0:1788845033.158365:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641578624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3942us (4217us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.158373:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 717 00000100:00100000:1.0:1788845033.167571:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.167671:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641578752 00000100:00100000:0.0:1788845033.167682:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641578752 00000100:00100000:0.0:1788845033.167757:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 718 00000100:00100000:0.0:1788845033.167783:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641578752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.167803:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.167821:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287ad28d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a94 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.167932:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660273920:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.168014:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.177687:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660273920:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845033.177743:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287ad28d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a94 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.177769:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287ad28d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a94 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.177783:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:0.0:1788845033.177854:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641578752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10073us (10293us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.177862:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 718 00000100:00100000:2.0:1788845033.189713:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.190055:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641578880 00000100:00100000:0.0:1788845033.190065:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641578880 00000100:00100000:0.0:1788845033.190158:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 719 00000100:00100000:0.0:1788845033.190169:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641578880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.190188:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.190207:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287ad294 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8a9b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.190279:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660274048:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.190372:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.192631:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660274048:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845033.192669:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287ad294 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a9b expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.192693:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287ad294 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a9b expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.192708:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864200, rc 0) 00000100:00100000:0.0:1788845033.192787:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641578880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2619us (3082us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.192797:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 719 00000100:00100000:3.0:1788845033.195575:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.196113:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641579008 00000100:00100000:2.0:1788845033.196123:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641579008 00000100:00100000:2.0:1788845033.196184:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 720 00000100:00100000:2.0:1788845033.196195:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641579008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845033.196213:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845033.196232:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad29b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8aa2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.196293:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660274176:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845033.196384:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.198981:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660274176:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845033.199021:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad29b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa2 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.199048:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287ad29b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa2 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.199063:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:2.0:1788845033.199165:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641579008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2970us (3597us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.199174:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 720 00000100:00100000:1.0:1788845033.203508:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.203675:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641579136 00000100:00100000:3.0:1788845033.203685:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641579136 00000100:00100000:3.0:1788845033.203752:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1048 00000100:00100000:3.0:1788845033.203763:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641579136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.203785:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.203788:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845033.203833:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287ad239 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a40 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.203862:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.203866:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845033.203931:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8aa9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.203957:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.203966:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.204000:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.204017:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 14 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.204030:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660274304:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845033.204104:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.208282:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.208370:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641579264 00000100:00100000:1.0:1788845033.208379:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641579264 00010000:00010000:1.0:1788845033.208415:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 15 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.208445:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 371 00000100:00100000:1.0:1788845033.208451:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641579264:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845033.208462:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845033.208482:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 15 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.208494:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 15 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.208511:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845033.208515:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.208523:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.208529:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.208555:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.208582:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660274432:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845033.208653:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f840 going to sleep for 11 seconds 00010000:00010000:3.0:1788845033.208974:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1ac00/0xb2187301287ad224 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a32 expref: 15 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.208987:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660274304:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845033.209009:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.209020:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.209023:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.209025:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845033.209076:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641579136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5315us (5579us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.209083:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1048 00000100:00100000:1.0:1788845033.211179:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660274432:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845033.211201:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845033.211235:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641579264:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2784us (2964us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.211243:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 371 00000100:00100000:1.0:1788845033.217985:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.218029:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.218124:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641579520 00000100:00100000:0.0:1788845033.218134:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641579520 00002000:00100000:0.0:1788845033.218203:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1421311) req@ffff94a03bea9f80 x1875739641579520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.218216:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03bea9f80 with x1875739641579520 ext(1048576->1421311) 00002000:00010000:0.0:1788845033.218222:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03bea9f80 00000100:00100000:0.0:1788845033.218248:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 378 00000100:00100000:0.0:1788845033.218269:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641579520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.220397:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641579392 00000100:00100000:3.0:1788845033.220410:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641579392 00002000:00100000:3.0:1788845033.220450:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea9c00 x1875739641579392/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.220462:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea9c00 with x1875739641579392 00000100:00100000:3.0:1788845033.220487:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 688 00000100:00100000:3.0:1788845033.220495:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641579392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00080000:1.0:1788845033.222474:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845033.230280:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641579520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12013us (12240us total) trans 0 rc 372736/372736 00000100:00100000:0.0:1788845033.230311:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 378 00002000:00100000:0.0:1788845033.230323:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1421311) req@ffff94a03bea9f80 x1875739641579520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 372736/372736 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.230333:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03bea9f80 with x1875739641579520 ext(1048576->1421311) 00002000:00010000:0.0:1788845033.230339:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03bea9f80 00000100:00100000:3.0:1788845033.233264:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641579392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12748us (15270us total) trans 0 rc 684032/684032 00000100:00100000:3.0:1788845033.233279:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 688 00002000:00100000:3.0:1788845033.233292:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea9c00 x1875739641579392/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 684032/684032 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.233302:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea9c00 with x1875739641579392 00000100:00100000:0.0:1788845033.241136:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.241235:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641579648 00000100:00100000:2.0:1788845033.241245:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641579648 00002000:00100000:2.0:1788845033.241293:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383c000 x1875739641579648/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.241304:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c000 with x1875739641579648 00000100:00100000:2.0:1788845033.241391:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 689 00000100:00100000:2.0:1788845033.241401:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641579648:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845033.248967:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 683971 00000004:00010000:0.0:1788845033.248977:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845032 -> 1788845033 00000004:00010000:0.0:1788845033.248980:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:0.0:1788845033.248982:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845032 -> 1788845033 00000004:00010000:0.0:1788845033.248984:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845033.249055:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641579648:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 7649us (7924us total) trans 4294967690 rc 0/0 00000100:00100000:0.0:1788845033.249066:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 689 00002000:00100000:0.0:1788845033.249080:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383c000 x1875739641579648/t4294967690(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/448 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.249090:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c000 with x1875739641579648 00000100:00100000:1.0:1788845033.251969:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.252109:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641579776 00000100:00100000:0.0:1788845033.252118:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641579776 00000100:00100000:0.0:1788845033.252181:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1049 00000100:00100000:0.0:1788845033.252192:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641579776:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845033.281965:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967690 is committed 00000100:00100000:2.0:1788845033.282493:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641579776:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 30295us (30525us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.282506:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1049 00000100:00100000:1.0:1788845033.286329:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.286534:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641580032 00000100:00100000:0.0:1788845033.286544:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641580032 00000100:00100000:0.0:1788845033.286639:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1050 00000100:00100000:0.0:1788845033.286658:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641580032:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845033.286712:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845033.286751:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad1e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a89f3 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.286769:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.286773:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845033.287096:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.287112:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.287120:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.287154:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.287171:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 10 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.287184:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383d880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660274560:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845033.287270:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.291616:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.291784:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383d880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660274560:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845033.291815:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.292120:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641580160 00000100:00100000:1.0:1788845033.292128:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641580160 00010000:00010000:1.0:1788845033.292164:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 11 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.292239:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 372 00000100:00100000:1.0:1788845033.292249:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641580160:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845033.292328:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845033.292338:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 11 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.292426:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 11 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.292474:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287ad232 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8a39 expref: 11 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.292488:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845033.292494:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.292505:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.292513:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.292544:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845033.292635:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641580160:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 386us (1028us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.292644:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 372 00010000:00010000:3.0:1788845033.292695:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.292702:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845033.292853:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967691 00000004:00010000:3.0:1788845033.292918:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 683971 00000004:00010000:3.0:1788845033.292922:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.292925:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845033.292927:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.292930:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 776 00000004:00010000:3.0:1788845033.292937:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e52da857 reply state 00000000d84b664b transno 4294967692 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.292947:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845033.293016:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641580032:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6360us (6695us total) trans 4294967692 rc 0/0 00000100:00100000:3.0:1788845033.293025:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1050 00000020:00080000:3.0:1788845033.293028:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788845033.297256:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845033.297561:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.297574:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.297598:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.297604:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031e6b000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845033.297609:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.297617:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.297635:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.297638:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845033.297643:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.297651:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad2a9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.298044:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641580288 00000100:00100000:3.0:1788845033.298055:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641580288 00000100:00100000:3.0:1788845033.298147:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1051 00000100:00100000:3.0:1788845033.298158:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641580288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.298184:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.298204:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ab0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.298232:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.298241:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.298284:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.298303:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 9 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.298316:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660274688:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845033.298410:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.303241:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660274688:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845033.303290:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.303309:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.303312:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.303316:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.303326:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e000, rc 0) 00000100:00100000:3.0:1788845033.303391:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641580288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5235us (6144us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.303400:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1051 00000100:00100000:0.0:1788845033.309439:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.309524:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641580416 00000100:00100000:2.0:1788845033.309534:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641580416 00010000:00010000:2.0:1788845033.309575:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 10 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.309643:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 373 00000100:00100000:2.0:1788845033.309654:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641580416:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845033.309669:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845033.309696:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 10 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.309800:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 10 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.309826:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad2a2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aa9 expref: 10 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.309852:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.309859:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.309870:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.310034:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.310075:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.310089:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660274816:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845033.310216:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.312911:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.312979:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641580544 00000100:00100000:3.0:1788845033.312987:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641580544 00000100:00100000:3.0:1788845033.313040:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 721 00000100:00100000:3.0:1788845033.313050:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641580544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.313085:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.313102:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ab7 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.313132:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.313140:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(2 11) 00010000:00010000:3.0:1788845033.313144:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:3.0:1788845033.313149:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.313182:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.313200:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 9 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.313214:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660274944:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845033.313291:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.316192:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660274944:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845033.316230:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.316252:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fe00, rc 0) 00000100:00100000:1.0:1788845033.316313:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641580544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3264us (3433us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.316340:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 721 00000100:00100000:2.0:1788845033.318300:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660274816:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845033.318327:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845033.318388:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641580416:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8736us (8959us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.318400:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 373 00000100:00100000:1.0:1788845033.320457:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.320541:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641580672 00000100:00100000:2.0:1788845033.320551:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641580672 00010000:00010000:2.0:1788845033.320585:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 10 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.320653:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 374 00000100:00100000:2.0:1788845033.320662:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641580672:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845033.320677:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845033.320700:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 10 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.320733:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 10 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.320755:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005093e963/0xb2187301287ad0b8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1642496->1654783) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a88e9 expref: 10 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.320771:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:2.0:1788845033.320776:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:2.0:1788845033.320784:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.320793:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.320830:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.320855:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660275072:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845033.321151:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.324267:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.324332:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.324404:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641580800 00000100:00100000:3.0:1788845033.324413:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641580800 00002000:00010000:3.0:1788845033.324460:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845033.324465:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7800 with x1875739641580800 00000100:00100000:3.0:1788845033.324499:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 690 00000100:00100000:3.0:1788845033.324509:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641580800:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845033.324910:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 683971 -> 187566 00000004:00010000:3.0:1788845033.324914:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.324916:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845033.324918:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.324920:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:0.0:1788845033.325048:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660275072:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845033.325070:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845033.325684:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641580800:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 1173us (1426us total) trans 4294967693 rc 0/0 00000100:00100000:3.0:1788845033.325693:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 690 00002000:00010000:3.0:1788845033.325698:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845033.325701:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7800 with x1875739641580800 00000100:00100000:0.0:1788845033.325784:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641580672:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5121us (5333us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.325792:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 374 00000100:00100000:1.0:1788845033.326153:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641580928 00000100:00100000:1.0:1788845033.326161:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641580928 00002000:00010000:1.0:1788845033.326193:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845033.326198:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02481c00 with x1875739641580928 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845033.326207:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02481c00. 00000100:00100000:1.0:1788845033.326237:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 379 00000100:00100000:1.0:1788845033.326245:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641580928:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845033.326463:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1421197 -> 0 00002000:00010000:1.0:1788845033.326467:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845032 -> 1788845033 00002000:00010000:1.0:1788845033.326470:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845031 -> 1788845029 00002000:00010000:1.0:1788845033.326472:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845032 -> 1788845033 00002000:00010000:1.0:1788845033.326474:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 15 -> 15 00000100:00100000:1.0:1788845033.326508:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641580928:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 263us (2175us total) trans 4294967425 rc 0/0 00000100:00100000:1.0:1788845033.326515:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 379 00002000:00010000:1.0:1788845033.326518:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845033.326521:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02481c00 with x1875739641580928 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845033.326526:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02481c00. 00000100:00100000:1.0:1788845033.330414:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.330500:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641581056 00000100:00100000:3.0:1788845033.330509:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641581056 00000100:00100000:3.0:1788845033.330571:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1052 00000100:00100000:3.0:1788845033.330582:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641581056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.330606:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.330625:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad2be lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8abe expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.330713:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.330856:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.330870:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845033.330925:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 187566 00000004:00200000:3.0:1788845033.330943:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845033.330964:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.330991:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad2be lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.331008:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.331017:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.331020:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845033.331024:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:3.0:1788845033.331081:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641581056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 501us (677us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.331089:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1052 00000100:00100000:2.0:1788845033.334661:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.334871:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641581184 00000100:00100000:0.0:1788845033.334880:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641581184 00000100:00100000:1.0:1788845033.335169:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1053 00000100:00100000:1.0:1788845033.335180:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641581184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.335199:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.335236:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ad2cc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ac5 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.335328:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b400/0xb2187301287ad2d3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.335354:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b400/0xb2187301287ad2d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.335373:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b400/0xb2187301287ad2d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845033.335400:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 187566 00000004:00200000:1.0:1788845033.335404:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845033.335422:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b400/0xb2187301287ad2d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.335449:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b800/0xb2187301287ad2cc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ac5 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.335467:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b400/0xb2187301287ad2d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ac5 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.335476:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845033.335479:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845033.335483:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b400, rc 0) 00000100:00100000:1.0:1788845033.335578:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641581184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 400us (927us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.335586:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1053 00000100:00100000:2.0:1788845033.338157:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.338198:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.338281:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641581312 00000100:00100000:0.0:1788845033.338291:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641581312 00000100:00100000:0.0:1788845033.338352:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1054 00000100:00100000:3.0:1788845033.338480:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641581440 00000100:00100000:3.0:1788845033.338484:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641581440 00000100:00100000:3.0:1788845033.338518:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 722 00000100:00100000:3.0:1788845033.338529:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641581440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.338548:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.338566:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad2da lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ad3 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.338624:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660275200:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.338702:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.338814:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641581312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.338825:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.338848:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad2e1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8acc expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.338877:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.338935:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660275328:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.338961:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845033.342054:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845031 00000100:00100000:2.0:1788845033.342084:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660275200:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845033.342118:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad2da lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ad3 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.342142:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad2da lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ad3 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.342156:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ea00, rc 0) 00000100:00100000:2.0:1788845033.342216:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641581440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3687us (4067us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.342224:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 722 00000100:00100000:2.0:1788845033.344067:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660275328:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845033.344093:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 187566 00010000:00010000:2.0:1788845033.344111:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad2e1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8acc expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.344129:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.344132:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845033.344137:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad2e1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8acc expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.344151:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a400, rc 0) 00000100:00100000:2.0:1788845033.344266:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641581312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5445us (6058us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.344275:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1054 00000100:00100000:0.0:1788845033.347549:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.347634:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.347702:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641581696 00000100:00100000:1.0:1788845033.347738:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641581696 00000100:00100000:1.0:1788845033.347793:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 723 00000100:00100000:1.0:1788845033.347803:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641581696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.347823:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.347849:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f1e54568/0xb2187301287ad2e8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ae1 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.347947:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660275456:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845033.348023:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.349961:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641581568 00000100:00100000:3.0:1788845033.349974:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641581568 00000100:00100000:2.0:1788845033.350235:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1055 00000100:00100000:2.0:1788845033.350246:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641581568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845033.350274:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845033.350294:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ad2ef lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ada expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.350329:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.350375:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660275584:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845033.350646:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.350946:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660275456:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.350974:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f1e54568/0xb2187301287ad2e8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae1 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.350995:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f1e54568/0xb2187301287ad2e8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae1 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.351007:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1aa00, rc 0) 00000100:00100000:3.0:1788845033.351044:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641581696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3242us (3409us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.351052:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 723 00000100:00100000:2.0:1788845033.354145:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660275584:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845033.354182:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 187566 00010000:00010000:2.0:1788845033.354204:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ad2ef lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ada expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.354226:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.354229:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845033.354234:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287ad2ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ada expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.354251:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e800, rc 0) 00000100:00100000:2.0:1788845033.354316:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641581568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4072us (6777us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.354325:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1055 00000100:00100000:1.0:1788845033.359677:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.360233:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641581952 00000100:00100000:2.0:1788845033.360245:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641581952 00000100:00100000:2.0:1788845033.360528:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1056 00000100:00100000:2.0:1788845033.360540:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641581952:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845033.360596:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845033.360624:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b400/0xb2187301287ad2d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ac5 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.360643:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.360647:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845033.361442:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.361463:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.361473:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.361508:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.361530:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 17 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.361545:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383d880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660275712:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845033.361699:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.369156:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.369520:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641582080 00000100:00100000:1.0:1788845033.369529:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641582080 00010000:00010000:1.0:1788845033.369569:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 18 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.369611:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 375 00000100:00100000:1.0:1788845033.369621:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641582080:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845033.369635:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845033.369642:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 18 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.369660:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 18 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.369681:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845033.369687:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.369725:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.369734:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.369754:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845033.369929:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641582080:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 307us (783us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.369937:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 375 00010000:00010000:3.0:1788845033.373244:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad2c5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8abe expref: 17 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.373346:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383d880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660275712:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845033.373433:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.373562:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.373570:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845033.373687:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967694 00000004:00010000:2.0:1788845033.373859:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 187566 -> 187566 00000004:00010000:2.0:1788845033.373863:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:2.0:1788845033.373866:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:2.0:1788845033.373868:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:2.0:1788845033.373870:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000004:00010000:2.0:1788845033.373876:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006ddd917a reply state 0000000013097234 transno 4294967695 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.374019:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845033.374183:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641581952:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13644us (14514us total) trans 4294967695 rc 0/0 00000100:00100000:2.0:1788845033.374191:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1056 00000020:00080000:2.0:1788845033.374197:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845033.376991:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.377505:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641582208 00000100:00100000:1.0:1788845033.377516:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641582208 00000100:00100000:1.0:1788845033.377583:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1057 00000100:00100000:1.0:1788845033.377594:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641582208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.377619:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.377641:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ae8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.377669:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.377680:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.377726:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.377744:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 16 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.377772:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660275840:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845033.377944:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9fcc0 going to sleep for 11 seconds 00010000:00010000:3.0:1788845033.378223:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.378232:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.378239:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.378245:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031e6b600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845033.378258:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.378266:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.378281:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.378284:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845033.378289:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.378295:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad2f6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.384071:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.384202:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660275840:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845033.384240:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.384325:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.384330:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845033.384335:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.384346:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b200, rc 0) 00000100:00100000:2.0:1788845033.384657:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641582208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7064us (7675us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.384668:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1057 00000100:00100000:3.0:1788845033.384798:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641582336 00000100:00100000:3.0:1788845033.384805:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641582336 00010000:00010000:3.0:1788845033.384847:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 17 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.384881:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 376 00000100:00100000:3.0:1788845033.385061:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d48850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641582336:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845033.385077:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845033.385103:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 17 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.385120:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 17 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.385145:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad2b0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab0 expref: 17 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.385183:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.385188:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.385198:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.385206:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.385241:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.385264:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660275968:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845033.385323:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.390211:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.390299:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641582464 00000100:00100000:1.0:1788845033.390309:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641582464 00000100:00100000:1.0:1788845033.390373:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 724 00000100:00100000:1.0:1788845033.390386:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641582464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.390407:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.390434:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8aef expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.390473:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.390483:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:1.0:1788845033.390487:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:1.0:1788845033.390603:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.390650:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.390671:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 8 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.390784:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660276096:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845033.390953:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.391149:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660275968:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845033.391175:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845033.391269:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d48850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641582336:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6210us (7207us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.391280:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 376 00000100:00100000:1.0:1788845033.395629:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.396119:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641582592 00000100:00100000:2.0:1788845033.396129:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641582592 00010000:00010000:2.0:1788845033.396168:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 9 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.396215:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 377 00000100:00100000:2.0:1788845033.396226:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641582592:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845033.396242:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845033.396276:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 9 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.396301:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 9 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.396326:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:2.0:1788845033.396330:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:2.0:1788845033.396339:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.396350:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.396400:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.396415:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660276224:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845033.396534:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00010000:00010000:0.0:1788845033.396967:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad2b7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ab7 expref: 9 pid: 6684 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.396984:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660276096:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845033.397012:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.397024:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1b600, rc 0) 00000100:00100000:0.0:1788845033.397056:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641582464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6673us (6857us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.397066:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 724 00000100:00100000:0.0:1788845033.400877:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.400990:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660276224:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845033.401016:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845033.401069:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641582592:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4844us (5448us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.401080:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 377 00000100:00100000:3.0:1788845033.401472:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641582720 00000100:00100000:3.0:1788845033.401481:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641582720 00002000:00010000:3.0:1788845033.401521:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845033.401526:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082680 with x1875739641582720 00000100:00100000:3.0:1788845033.401565:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 691 00000100:00100000:3.0:1788845033.401574:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641582720:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845033.401787:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 187566 -> 351819 00000004:00010000:3.0:1788845033.401791:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.401806:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845033.401809:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.401812:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:3.0:1788845033.401862:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641582720:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 289us (997us total) trans 4294967696 rc 0/0 00000100:00100000:3.0:1788845033.401869:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 691 00002000:00010000:3.0:1788845033.401873:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845033.401875:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082680 with x1875739641582720 00000100:00100000:1.0:1788845033.404309:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.404389:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641582848 00000100:00100000:0.0:1788845033.404399:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641582848 00002000:00010000:0.0:1788845033.404435:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845033.404440:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360a80 with x1875739641582848 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845033.404451:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360a80. 00000100:00100000:0.0:1788845033.404484:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 380 00000100:00100000:0.0:1788845033.404494:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641582848:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845033.404724:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1788845033.404728:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00002000:00010000:0.0:1788845033.404744:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845031 -> 1788845029 00002000:00010000:0.0:1788845033.404747:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00002000:00010000:0.0:1788845033.404750:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 15 -> 15 00000100:00100000:0.0:1788845033.404813:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641582848:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 319us (510us total) trans 4294967426 rc 0/0 00000100:00100000:0.0:1788845033.404821:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 380 00002000:00010000:0.0:1788845033.404825:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845033.404828:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360a80 with x1875739641582848 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845033.404833:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360a80. 00000100:00100000:1.0:1788845033.408127:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.408226:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641582976 00000100:00100000:0.0:1788845033.408236:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641582976 00000100:00100000:0.0:1788845033.408296:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1058 00000100:00100000:0.0:1788845033.408307:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641582976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.408333:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.408352:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad30b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8af6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.408455:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.408472:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.408485:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845033.408513:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 351819 00000004:00200000:0.0:1788845033.408518:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845033.408536:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.408580:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad30b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.408600:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.408610:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.408613:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845033.408617:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a800, rc 0) 00000100:00100000:0.0:1788845033.408678:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641582976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 372us (560us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.408686:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1058 00000100:00100000:2.0:1788845033.414563:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.414751:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641583104 00000100:00100000:0.0:1788845033.414762:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641583104 00000100:00100000:0.0:1788845033.414826:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1059 00000100:00100000:0.0:1788845033.414845:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641583104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.414869:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.414907:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8afd expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.414938:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.414950:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.415009:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.415028:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 13 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.415042:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660276352:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845033.415122:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.418933:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.419036:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641583232 00000100:00100000:2.0:1788845033.419046:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641583232 00010000:00010000:2.0:1788845033.419087:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 14 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.419132:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 378 00000100:00100000:2.0:1788845033.419142:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641583232:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845033.419158:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845033.419187:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 14 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.419204:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 14 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.419226:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.419233:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.419244:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.419253:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.419293:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.419308:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d92a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660276480:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845033.419395:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00010000:00010000:0.0:1788845033.420493:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01e1b200/0xb2187301287ad2fd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ae8 expref: 14 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.420514:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660276352:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845033.420541:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.420555:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.420559:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.420562:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b400, rc 0) 00000100:00100000:0.0:1788845033.420618:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641583104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5783us (6069us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.420627:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1059 00000100:00100000:3.0:1788845033.422301:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d92a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660276480:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845033.422326:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845033.422387:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641583232:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3246us (3464us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.422397:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 378 00000100:00100000:0.0:1788845033.426597:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.426740:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641583360 00000100:00100000:3.0:1788845033.426749:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641583360 00000100:00100000:3.0:1788845033.426808:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 725 00000100:00100000:3.0:1788845033.426819:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641583360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.426847:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.426866:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b04 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.426934:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.426943:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:3.0:1788845033.426948:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:3.0:1788845033.426968:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1052671] (req 1048576->1052671) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.427003:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.427021:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 10 pid: 7983 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.427035:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660276608:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845033.427110:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.430291:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660276608:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845033.430330:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1052671] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.430351:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e000, rc 0) 00000100:00100000:3.0:1788845033.430475:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641583360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3656us (3885us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.430484:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 725 00000100:00100000:2.0:1788845033.431528:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.431634:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641583488 00000100:00100000:1.0:1788845033.431642:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641583488 00010000:00010000:1.0:1788845033.431678:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 11 pid: 7983 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.431723:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 379 00000100:00100000:1.0:1788845033.431732:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641583488:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845033.431745:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845033.431768:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 11 pid: 7983 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.431784:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 11 pid: 7983 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.431808:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b397b794/0xb2187301287ad304 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8aef expref: 11 pid: 7983 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.431824:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:1.0:1788845033.431852:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:1.0:1788845033.431862:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.431872:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.431943:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.431958:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02362d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660276736:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845033.432018:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.438041:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.438069:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.438173:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641583616 00000100:00100000:0.0:1788845033.438185:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641583616 00000100:00100000:0.0:1788845033.438253:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1060 00000100:00100000:0.0:1788845033.438289:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641583616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.438311:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.438336:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad327 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b0b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.438390:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.438439:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660276864:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.438515:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.438959:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641583744 00000100:00100000:3.0:1788845033.438964:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641583744 00000100:00100000:3.0:1788845033.439006:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 726 00000100:00100000:3.0:1788845033.439014:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641583744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.439024:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.439036:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad32e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b12 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.439080:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660276992:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.439117:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.439211:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02362d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660276736:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845033.439231:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845033.439260:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641583488:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7529us (7736us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.439270:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 379 00002000:00010000:3.0:1788845033.442232:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1049539 00002000:00010000:3.0:1788845033.442242:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845031 00000100:00100000:3.0:1788845033.442269:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660276992:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.442304:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad32e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b12 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.442329:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad32e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b12 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.442342:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:3.0:1788845033.442408:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641583744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3394us (4337us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.442417:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 726 00000004:00010000:0.0:1788845033.442949:0:5839:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 351819 -> 1048576 00000100:00100000:0.0:1788845033.442970:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660276864:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845033.442984:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845033.442997:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad327 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b0b expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.443011:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.443014:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.443017:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad327 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b0b expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.443028:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a400, rc 0) 00000100:00100000:0.0:1788845033.443071:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641583616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4785us (5043us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.443078:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1060 00000100:00100000:1.0:1788845033.447149:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.447235:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641583872 00000100:00100000:3.0:1788845033.447247:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641583872 00000100:00100000:3.0:1788845033.447319:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1061 00000100:00100000:3.0:1788845033.447331:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641583872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.447366:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.447390:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad335 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b19 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.447427:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.447485:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660277120:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.447570:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.450245:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.450386:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641584000 00000100:00100000:1.0:1788845033.450396:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641584000 00000100:00100000:1.0:1788845033.450452:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 727 00000100:00100000:1.0:1788845033.450463:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641584000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.450482:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.450500:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019a90c7b/0xb2187301287ad33c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b20 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.450558:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660277248:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845033.450641:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f480 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.455009:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660277248:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845033.455050:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019a90c7b/0xb2187301287ad33c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b20 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.455074:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019a90c7b/0xb2187301287ad33c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b20 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.455087:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01e1a400, rc 0) 00000100:00100000:2.0:1788845033.455147:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641584000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4686us (4911us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.455156:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 727 00000100:00100000:3.0:1788845033.455782:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660277120:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845033.455798:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845033.455812:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad335 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b19 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.455828:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.455831:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.455844:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad335 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b19 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.455856:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:3.0:1788845033.455923:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641583872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8594us (8787us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.455932:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1061 00000100:00100000:1.0:1788845033.463200:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.463275:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.463536:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641584256 00000100:00100000:0.0:1788845033.463545:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641584256 00000100:00100000:0.0:1788845033.463611:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 728 00000100:00100000:0.0:1788845033.463620:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641584256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.463641:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.463660:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000015095dc9/0xb2187301287ad343 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b2e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.463728:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660277376:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.463815:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.464000:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641584128 00000100:00100000:3.0:1788845033.464005:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641584128 00000100:00100000:3.0:1788845033.464035:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1062 00000100:00100000:3.0:1788845033.464042:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641584128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.464053:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.464066:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad34a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.464096:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.464130:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660277504:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.464162:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.466489:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660277504:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845033.466517:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845033.466536:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad34a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b27 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.466555:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.466558:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.466563:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad34a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b27 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.466576:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:3.0:1788845033.466632:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641584128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2590us (3442us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.466640:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1062 00000100:00100000:0.0:1788845033.466752:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660277376:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845033.466772:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000015095dc9/0xb2187301287ad343 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b2e expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.466789:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000015095dc9/0xb2187301287ad343 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b2e expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.466800:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a400, rc 0) 00000100:00100000:0.0:1788845033.466826:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641584256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3206us (3550us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.466832:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 728 00000100:00100000:1.0:1788845033.470284:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.470371:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641584384 00000100:00100000:0.0:1788845033.470381:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641584384 00000100:00100000:0.0:1788845033.470438:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1063 00000100:00100000:0.0:1788845033.470449:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641584384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.470470:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.470491:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b35 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.470536:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.470547:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b35 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.470580:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.470598:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 16 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.470611:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660277632:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845033.470690:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.474557:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.474631:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641584512 00000100:00100000:3.0:1788845033.474640:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641584512 00002000:00100000:3.0:1788845033.474690:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02361f80 x1875739641584512/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.474702:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02361f80 with x1875739641584512 00010000:00010000:3.0:1788845033.474711:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 17 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.474741:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 17 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845033.474763:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f02361f80 00000100:00100000:3.0:1788845033.474801:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 24 00000100:00100000:3.0:1788845033.474811:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641584512:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845033.475144:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660277632:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845033.475174:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b35 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.475191:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.475195:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.475200:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b35 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.475210:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a200, rc 0) 00000100:00100000:0.0:1788845033.475242:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641584384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4796us (4968us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.475256:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1063 00000004:00010000:3.0:1788845033.479855:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845033.479865:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.479868:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845033.479870:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.479872:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:3.0:1788845033.479998:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641584512:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5182us (5446us total) trans 4294967697 rc 0/0 00000100:00100000:3.0:1788845033.480011:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 24 00002000:00100000:3.0:1788845033.480024:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02361f80 x1875739641584512/t4294967697(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/448 e 0 to 0 dl 1788845044 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.480035:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02361f80 with x1875739641584512 00010000:00010000:3.0:1788845033.480045:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 19 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.480065:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 19 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845033.480073:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f02361f80 00000100:00100000:1.0:1788845033.483607:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.484406:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641584640 00000100:00100000:3.0:1788845033.484417:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641584640 00010000:00010000:3.0:1788845033.484464:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 19 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.484530:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 380 00000100:00100000:3.0:1788845033.484540:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641584640:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845033.484556:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845033.484584:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 19 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.484602:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 19 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.484622:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b400/0xb2187301287ad319 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8afd expref: 19 pid: 11036 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.484638:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.484645:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b35 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.484656:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b35 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.484666:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b35 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.484702:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b35 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.485143:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660277760:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845033.485355:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.489496:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.489614:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660277760:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845033.489639:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845033.489700:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641584640:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5161us (6102us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.489721:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 380 00000100:00100000:3.0:1788845033.489811:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641584768 00000100:00100000:3.0:1788845033.489819:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641584768 00002000:00100000:3.0:1788845033.489875:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93100 x1875739641584768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.489925:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93100 with x1875739641584768 00000100:00100000:3.0:1788845033.489951:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 692 00000100:00100000:3.0:1788845033.489960:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641584768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.490163:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641584768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 203us (677us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845033.490171:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 692 00002000:00100000:3.0:1788845033.490177:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93100 x1875739641584768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.490186:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93100 with x1875739641584768 00000100:00100000:2.0:1788845033.493971:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.494186:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641584896 00000100:00100000:1.0:1788845033.494197:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641584896 00002000:00100000:1.0:1788845033.494252:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d92300 x1875739641584896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.494275:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d92300 with x1875739641584896 00000100:00100000:1.0:1788845033.494313:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 693 00000100:00100000:1.0:1788845033.494323:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641584896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845033.494569:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641584896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 246us (607us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845033.494577:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 693 00002000:00100000:1.0:1788845033.494585:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d92300 x1875739641584896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.494592:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d92300 with x1875739641584896 00000100:00100000:2.0:1788845033.498457:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.498556:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641585024 00000100:00100000:3.0:1788845033.498567:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641585024 00002000:00100000:3.0:1788845033.498617:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91880 x1875739641585024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.498628:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91880 with x1875739641585024 00000100:00100000:3.0:1788845033.498666:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 694 00000100:00100000:3.0:1788845033.498676:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.498953:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (506us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845033.498978:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 694 00002000:00100000:3.0:1788845033.498986:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91880 x1875739641585024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.498994:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91880 with x1875739641585024 00000100:00100000:2.0:1788845033.502159:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.502260:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641585152 00000100:00100000:1.0:1788845033.502269:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641585152 00002000:00100000:1.0:1788845033.502315:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91180 x1875739641585152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.502325:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91180 with x1875739641585152 00000100:00100000:1.0:1788845033.502370:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 695 00000100:00100000:1.0:1788845033.502380:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641585152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845033.502585:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641585152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 206us (435us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845033.502608:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 695 00002000:00100000:1.0:1788845033.502616:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91180 x1875739641585152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.502623:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91180 with x1875739641585152 00000100:00100000:2.0:1788845033.506458:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.506684:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641585280 00000100:00100000:3.0:1788845033.506695:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641585280 00002000:00100000:3.0:1788845033.506811:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90700 x1875739641585280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.506823:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90700 with x1875739641585280 00000100:00100000:2.0:1788845033.507088:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 696 00000100:00100000:2.0:1788845033.507099:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845033.507319:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 221us (869us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845033.507327:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 696 00002000:00100000:2.0:1788845033.507335:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90700 x1875739641585280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.507343:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90700 with x1875739641585280 00000100:00100000:1.0:1788845033.510053:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.510144:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641585408 00000100:00100000:3.0:1788845033.510153:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641585408 00002000:00100000:3.0:1788845033.510204:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03447b80 x1875739641585408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.510217:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03447b80 with x1875739641585408 00000100:00100000:3.0:1788845033.510257:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 697 00000100:00100000:3.0:1788845033.510268:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.510500:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (455us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845033.510508:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 697 00002000:00100000:3.0:1788845033.510515:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03447b80 x1875739641585408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.510523:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03447b80 with x1875739641585408 00000100:00100000:1.0:1788845033.513864:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.514008:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641585536 00000100:00100000:0.0:1788845033.514018:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641585536 00002000:00100000:0.0:1788845033.514068:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03444700 x1875739641585536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.514081:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03444700 with x1875739641585536 00000100:00100000:0.0:1788845033.514107:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 698 00000100:00100000:0.0:1788845033.514116:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641585536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.514499:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641585536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 381us (647us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.514507:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 698 00002000:00100000:0.0:1788845033.514515:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03444700 x1875739641585536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.514522:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03444700 with x1875739641585536 00000100:00100000:1.0:1788845033.517634:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.517777:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641585664 00000100:00100000:0.0:1788845033.517787:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641585664 00002000:00100000:0.0:1788845033.517847:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03444e00 x1875739641585664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.517858:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03444e00 with x1875739641585664 00000100:00100000:0.0:1788845033.518022:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 699 00000100:00100000:0.0:1788845033.518033:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585664:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.518402:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585664:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 368us (774us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.518411:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 699 00002000:00100000:0.0:1788845033.518418:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03444e00 x1875739641585664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.518426:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03444e00 with x1875739641585664 00000100:00100000:1.0:1788845033.522081:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.522180:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641585792 00000100:00100000:0.0:1788845033.522189:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641585792 00002000:00100000:0.0:1788845033.522240:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03447480 x1875739641585792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.522251:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03447480 with x1875739641585792 00000100:00100000:0.0:1788845033.522329:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 700 00000100:00100000:0.0:1788845033.522340:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641585792:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.522632:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641585792:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (560us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.522641:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 700 00002000:00100000:0.0:1788845033.522648:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03447480 x1875739641585792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.522655:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03447480 with x1875739641585792 00000100:00100000:1.0:1788845033.525423:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.525738:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641585920 00000100:00100000:2.0:1788845033.525747:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641585920 00002000:00100000:2.0:1788845033.525801:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a039cdac50 x1875739641585920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.525812:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a039cdac50 with x1875739641585920 00000100:00100000:2.0:1788845033.525862:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 701 00000100:00100000:2.0:1788845033.525872:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a039cdac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845033.526155:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a039cdac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641585920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 283us (742us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845033.526163:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 701 00002000:00100000:2.0:1788845033.526170:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a039cdac50 x1875739641585920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.526178:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a039cdac50 with x1875739641585920 00000100:00100000:2.0:1788845033.532291:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.532380:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641586048 00000100:00100000:0.0:1788845033.532389:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641586048 00002000:00100000:0.0:1788845033.532442:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93b80 x1875739641586048/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.532473:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93b80 with x1875739641586048 00000100:00100000:0.0:1788845033.532515:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 702 00000100:00100000:0.0:1788845033.532526:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641586048:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.532830:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641586048:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 304us (547us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.532847:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 702 00002000:00100000:0.0:1788845033.532855:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93b80 x1875739641586048/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.532863:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93b80 with x1875739641586048 00000100:00100000:2.0:1788845033.535632:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.535785:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641586176 00000100:00100000:3.0:1788845033.535795:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641586176 00002000:00100000:3.0:1788845033.535877:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93100 x1875739641586176/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.535925:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93100 with x1875739641586176 00000100:00100000:3.0:1788845033.535955:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 703 00000100:00100000:3.0:1788845033.535964:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641586176:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.536219:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641586176:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 255us (595us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845033.536228:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 703 00002000:00100000:3.0:1788845033.536235:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93100 x1875739641586176/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.536242:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93100 with x1875739641586176 00000100:00100000:3.0:1788845033.539958:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.540104:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641586304 00000100:00100000:2.0:1788845033.540114:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641586304 00002000:00100000:2.0:1788845033.540165:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb2a00 x1875739641586304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.540177:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb2a00 with x1875739641586304 00000100:00100000:2.0:1788845033.540218:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 704 00000100:00100000:2.0:1788845033.540228:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641586304:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845033.540498:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641586304:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 271us (551us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845033.540507:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 704 00002000:00100000:2.0:1788845033.540514:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb2a00 x1875739641586304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.540521:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb2a00 with x1875739641586304 00000100:00100000:2.0:1788845033.543122:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.543461:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641586432 00000100:00100000:0.0:1788845033.543470:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641586432 00002000:00100000:0.0:1788845033.543514:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d92300 x1875739641586432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.543525:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d92300 with x1875739641586432 00000100:00100000:0.0:1788845033.543560:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 705 00000100:00100000:0.0:1788845033.543568:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641586432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.543957:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641586432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 388us (842us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.543965:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 705 00002000:00100000:0.0:1788845033.543972:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d92300 x1875739641586432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.543980:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d92300 with x1875739641586432 00000100:00100000:1.0:1788845033.547670:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.547823:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641586560 00000100:00100000:2.0:1788845033.547841:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641586560 00002000:00100000:2.0:1788845033.547967:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03446680 x1875739641586560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.547979:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03446680 with x1875739641586560 00000100:00100000:2.0:1788845033.548051:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 706 00000100:00100000:2.0:1788845033.548062:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641586560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845033.548294:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03446680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641586560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (634us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845033.548302:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 706 00002000:00100000:2.0:1788845033.548310:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03446680 x1875739641586560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.548324:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03446680 with x1875739641586560 00000100:00100000:3.0:1788845033.551062:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.551163:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641586688 00000100:00100000:0.0:1788845033.551190:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641586688 00000100:00100000:0.0:1788845033.551260:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1064 00000100:00100000:0.0:1788845033.551271:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641586688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.551300:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.551327:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad358 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b3c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.551434:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad35f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.551449:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad35f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.551460:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad35f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845033.551486:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 351819 00000004:00200000:0.0:1788845033.551491:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845033.551507:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad35f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.551533:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad358 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b3c expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.551550:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad35f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b3c expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.551558:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.551560:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845033.551563:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b000, rc 0) 00000100:00100000:0.0:1788845033.551733:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641586688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 462us (679us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.551740:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1064 00000100:00100000:1.0:1788845033.554488:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.554581:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641586816 00000100:00100000:3.0:1788845033.554590:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641586816 00000100:00100000:3.0:1788845033.554682:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1065 00000100:00100000:3.0:1788845033.554692:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641586816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.554725:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.554747:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b43 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.554784:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845033.554797:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845033.554805:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.554832:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.554863:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.554867:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:3.0:1788845033.554946:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641586816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 254us (465us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.554952:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1065 00000100:00100000:1.0:1788845033.565334:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.565438:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641586944 00000100:00100000:3.0:1788845033.565449:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641586944 00002000:00100000:3.0:1788845033.565500:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445c00 x1875739641586944/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.565511:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445c00 with x1875739641586944 00000100:00100000:3.0:1788845033.565553:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 707 00000100:00100000:3.0:1788845033.565563:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641586944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.579794:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641586944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14220us (14461us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1788845033.579809:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 707 00002000:00100000:3.0:1788845033.579824:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445c00 x1875739641586944/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.579842:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445c00 with x1875739641586944 00000100:00100000:1.0:1788845033.585128:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.585229:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641587072 00000100:00100000:3.0:1788845033.585237:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641587072 00000100:00100000:3.0:1788845033.585297:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 729 00000100:00100000:3.0:1788845033.585307:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641587072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.585342:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.585371:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad36d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b4a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.585430:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660277888:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.585677:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788845033.588274:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845031 -> 1788845033 00000100:00100000:3.0:1788845033.588309:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660277888:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.588349:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad36d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b4a expref: 11 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.588452:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad36d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b4a expref: 11 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.588468:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:3.0:1788845033.588570:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641587072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3263us (3454us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.588579:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 729 00000100:00100000:1.0:1788845033.591523:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.591619:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641587200 00000100:00100000:2.0:1788845033.591629:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641587200 00000100:00100000:2.0:1788845033.591731:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 730 00000100:00100000:2.0:1788845033.591741:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641587200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845033.591760:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845033.591777:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad374 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b51 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.591855:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d91880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660278016:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845033.592063:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.594613:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d91880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660278016:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845033.594651:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad374 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b51 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.594675:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287ad374 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b51 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.594690:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845033.594857:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641587200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3117us (3342us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.594866:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 730 00000100:00100000:1.0:1788845033.598149:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.598242:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641587328 00000100:00100000:3.0:1788845033.598251:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641587328 00000100:00100000:3.0:1788845033.598517:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1066 00000100:00100000:3.0:1788845033.598529:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641587328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.598552:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.598556:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845033.598601:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a200/0xb2187301287ad351 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b35 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.598636:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.598641:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845033.598656:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.598681:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.598689:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.598733:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.598751:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 19 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.598764:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660278144:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845033.598857:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.603653:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.603730:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641587456 00000100:00100000:2.0:1788845033.603739:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641587456 00010000:00010000:2.0:1788845033.603780:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 20 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.603823:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 381 00000100:00100000:2.0:1788845033.603840:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641587456:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845033.603855:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845033.603932:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 20 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.603951:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 20 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.603995:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.604002:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.604014:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.604024:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.604061:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.604076:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d91880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660278272:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845033.604153:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788845033.604244:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad366 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b43 expref: 20 pid: 5840 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.604745:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660278144:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845033.604776:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.604811:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.604814:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.604816:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ec00, rc 0) 00000100:00100000:3.0:1788845033.604863:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03446300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641587328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6337us (6723us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.604871:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1066 00000100:00100000:1.0:1788845033.610809:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.610867:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641587584 00000100:00100000:3.0:1788845033.610877:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641587584 00000100:00100000:3.0:1788845033.611105:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 708 00000100:00100000:3.0:1788845033.611117:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641587584:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845033.611378:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845033.611382:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.611385:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845031 -> 1788845031 00000004:00010000:3.0:1788845033.611387:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.611389:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:3.0:1788845033.611464:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641587584:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 349us (664us total) trans 4294967698 rc 0/0 00000100:00100000:3.0:1788845033.611471:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 708 00000100:00100000:2.0:1788845033.613667:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d91880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660278272:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845033.613693:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845033.617235:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.617422:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641587712 00000100:00100000:0.0:1788845033.617433:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641587712 00000100:00100000:2.0:1788845033.617470:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03446a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641587456:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 13636us (13824us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.617483:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 381 00000100:00100000:0.0:1788845033.617496:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1067 00000100:00100000:0.0:1788845033.617507:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641587712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.617528:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.617549:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad382 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b5f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.617591:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.618006:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660278400:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.618051:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000004:00010000:0.0:1788845033.624132:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845031 -> 1788845033 00000100:00100000:0.0:1788845033.624164:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660278400:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845033.624186:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845033.624206:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad382 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b5f expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.624227:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.624231:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.624236:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a400/0xb2187301287ad382 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b5f expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.624269:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a400, rc 0) 00000100:00100000:0.0:1788845033.624371:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641587712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6865us (7144us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.624381:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1067 00000100:00100000:1.0:1788845033.632076:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.632185:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641587840 00000100:00100000:0.0:1788845033.632195:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641587840 00000100:00100000:0.0:1788845033.632297:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1068 00000100:00100000:0.0:1788845033.632308:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641587840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.632329:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.632350:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad389 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b66 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.632399:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.632439:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660278528:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.632520:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.638916:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660278528:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845033.638943:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845033.638964:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad389 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b66 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.638985:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.638987:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.638992:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b600/0xb2187301287ad389 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b66 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.639021:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6b600, rc 0) 00000100:00100000:0.0:1788845033.639085:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641587840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6778us (7020us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.639094:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1068 00000100:00100000:2.0:1788845033.642188:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.642312:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641587968 00000100:00100000:1.0:1788845033.642321:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641587968 00000100:00100000:1.0:1788845033.642412:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 731 00000100:00100000:1.0:1788845033.642424:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641587968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.642441:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.642459:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ad390 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b6d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.642540:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660278656:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845033.642623:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.645824:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660278656:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845033.645877:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ad390 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b6d expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.646016:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ad390 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b6d expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.646029:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:2.0:1788845033.646129:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641587968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3706us (3950us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.646137:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 731 00000100:00100000:2.0:1788845033.649702:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.649790:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641588096 00000100:00100000:1.0:1788845033.649799:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641588096 00000100:00100000:1.0:1788845033.649865:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 732 00000100:00100000:1.0:1788845033.649876:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641588096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.649931:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.649951:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad397 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b74 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.650011:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660278784:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845033.650085:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.652327:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660278784:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845033.652582:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad397 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b74 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.652609:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad397 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b74 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.652623:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845033.652683:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641588096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2808us (2989us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845033.652692:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 732 00000100:00100000:2.0:1788845033.656288:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.656368:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641588224 00000100:00100000:3.0:1788845033.656377:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641588224 00000100:00100000:3.0:1788845033.656440:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1069 00000100:00100000:3.0:1788845033.656450:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641588224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.656471:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.656492:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b7b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.656536:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.656546:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.656579:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.656597:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 14 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.656611:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660278912:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845033.656690:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.659148:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660278912:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845033.659189:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.659210:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.659213:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.659217:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.659227:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:3.0:1788845033.659300:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641588224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2851us (3021us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.659325:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1069 00000100:00100000:2.0:1788845033.662196:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.662287:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641588352 00000100:00100000:3.0:1788845033.662297:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641588352 00010000:00010000:3.0:1788845033.662338:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 15 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.662381:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 382 00000100:00100000:3.0:1788845033.662391:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641588352:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845033.662406:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845033.662435:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 15 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.662452:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 15 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.662474:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad37b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b58 expref: 15 pid: 5841 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.662488:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.662493:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.662504:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.662512:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.662562:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.662575:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660279040:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845033.662693:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.666013:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.666103:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641588480 00000100:00100000:1.0:1788845033.666113:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641588480 00000100:00100000:1.0:1788845033.666188:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 709 00000100:00100000:1.0:1788845033.666198:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641588480:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845033.666489:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788845033.666493:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:1.0:1788845033.666496:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:1.0:1788845033.666498:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:1.0:1788845033.666500:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:1.0:1788845033.666565:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641588480:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 368us (562us total) trans 4294967699 rc 0/0 00000100:00100000:1.0:1788845033.666573:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 709 00000100:00100000:3.0:1788845033.667676:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660279040:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845033.667699:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845033.667757:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641588352:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5367us (5569us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.667767:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 382 00000100:00100000:2.0:1788845033.675224:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.675567:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641588736 00000100:00100000:3.0:1788845033.675577:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641588736 00000100:00100000:3.0:1788845033.675640:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1070 00000100:00100000:3.0:1788845033.675651:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641588736:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845033.675707:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845033.675747:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b000/0xb2187301287ad35f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b3c expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.675765:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.675769:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845033.676033:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.676048:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.676076:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.676108:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.676127:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 13 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.676141:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660279168:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845033.676228:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.679458:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660279168:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845033.679490:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.681332:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.681464:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641588864 00000100:00100000:1.0:1788845033.681476:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641588864 00010000:00010000:1.0:1788845033.681517:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 14 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.681561:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 383 00000100:00100000:1.0:1788845033.681571:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d4a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641588864:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845033.681585:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845033.681592:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 14 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.681610:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 14 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.681631:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a800/0xb2187301287ad312 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8af6 expref: 14 pid: 5839 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.681659:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845033.681664:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.681672:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.681680:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.681717:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845033.681787:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d4a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641588864:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 216us (462us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.681806:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 383 00010000:00010000:3.0:1788845033.682057:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.682066:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845033.682173:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967700 00000004:00010000:3.0:1788845033.682218:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788845033.682222:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.682225:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.682227:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:3.0:1788845033.682230:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000004:00010000:3.0:1788845033.682244:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eda0a0f1 reply state 0000000015499654 transno 4294967701 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.682252:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845033.682325:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641588736:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6676us (7111us total) trans 4294967701 rc 0/0 00000100:00100000:3.0:1788845033.682332:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1070 00000020:00080000:3.0:1788845033.682336:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845033.684727:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.684742:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.684750:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.684756:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191ec00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845033.684761:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.684767:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.684784:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.684787:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845033.684799:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.684806:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad3a5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.689489:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.689569:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641588992 00000100:00100000:0.0:1788845033.689593:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641588992 00002000:00010000:0.0:1788845033.689633:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1445258->18446744073709551615) 00002000:00100000:0.0:1788845033.689638:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d92d80 with x1875739641588992 ext(1445258->18446744073709551615) 00002000:00010000:0.0:1788845033.689649:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d92d80. 00000100:00100000:0.0:1788845033.689687:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 381 00000100:00100000:0.0:1788845033.689697:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641588992:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845033.689994:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1049539 -> 1445258 00002000:00010000:0.0:1788845033.689999:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00002000:00010000:0.0:1788845033.690001:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845033 -> 1788845029 00002000:00010000:0.0:1788845033.690003:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00002000:00010000:0.0:1788845033.690006:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 15 -> 15 00000100:00100000:0.0:1788845033.690068:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641588992:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 372us (589us total) trans 4294967427 rc 0/0 00000100:00100000:0.0:1788845033.690075:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 381 00002000:00010000:0.0:1788845033.690079:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1445258->18446744073709551615) 00002000:00100000:0.0:1788845033.690082:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d92d80 with x1875739641588992 ext(1445258->18446744073709551615) 00002000:00010000:0.0:1788845033.690087:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d92d80. 00000100:00100000:1.0:1788845033.700489:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.700543:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.700681:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641589120 00000100:00100000:2.0:1788845033.700691:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641589120 00002000:00100000:2.0:1788845033.700754:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1052672->1445887) req@ffff949f03446a00 x1875739641589120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.700765:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03446a00 with x1875739641589120 ext(1052672->1445887) 00002000:00010000:2.0:1788845033.700773:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03446a00 00000100:00100000:2.0:1788845033.700816:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 382 00000100:00100000:2.0:1788845033.700827:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2368:x1875739641589120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.703851:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641589248 00000100:00100000:0.0:1788845033.703865:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641589248 00002000:00100000:0.0:1788845033.703953:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445500 x1875739641589248/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.703966:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445500 with x1875739641589248 00000100:00100000:0.0:1788845033.704007:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 710 00000100:00100000:0.0:1788845033.704019:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:2368:x1875739641589248:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.708617:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03446a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2368:x1875739641589120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 7780us (8127us total) trans 0 rc 393216/393216 00000100:00100000:3.0:1788845033.708632:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 382 00002000:00100000:3.0:1788845033.708645:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1052672->1445887) req@ffff949f03446a00 x1875739641589120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 393216/393216 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.708655:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03446a00 with x1875739641589120 ext(1052672->1445887) 00002000:00010000:3.0:1788845033.708662:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03446a00 00000100:00100000:1.0:1788845033.716507:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.716576:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641589376 00000100:00100000:3.0:1788845033.716584:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641589376 00002000:00100000:3.0:1788845033.716626:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1445257) req@ffff949f03447100 x1875739641589376/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 504/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.716634:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03447100 with x1875739641589376 ext(1048576->1445257) 00002000:00010000:3.0:1788845033.716638:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03447100 00000100:00100000:3.0:1788845033.716666:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 383 00000100:00100000:3.0:1788845033.716672:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641589376:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845033.720523:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:2369:x1875739641589376:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3842us (4015us total) trans 4294967428 rc 0/0 00000100:00100000:0.0:1788845033.720541:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 383 00002000:00100000:0.0:1788845033.720552:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1445257) req@ffff949f03447100 x1875739641589376/t4294967428(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 504/448 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.720559:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03447100 with x1875739641589376 ext(1048576->1445257) 00002000:00010000:0.0:1788845033.720563:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03447100 00000100:00100000:0.0:1788845033.726505:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.733252:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641589504 00000100:00100000:3.0:1788845033.733268:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641589504 00000100:00100000:2.0:1788845033.733439:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 733 00000100:00100000:2.0:1788845033.733450:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2367:x1875739641589504:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:0.0:1788845033.739844:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:2368:x1875739641589248:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 35802us (39275us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1788845033.739860:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 710 00002000:00100000:0.0:1788845033.739873:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445500 x1875739641589248/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.739954:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445500 with x1875739641589248 00000100:00100000:2.0:1788845033.768991:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2367:x1875739641589504:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 35531us (42488us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.769005:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 733 00000001:00080000:1.0:1788845033.769214:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967428 is committed 00000001:00080000:1.0:1788845033.769233:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967426 is committed 00000100:00100000:1.0:1788845033.772324:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.772467:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641589632 00000100:00100000:0.0:1788845033.772476:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641589632 00000100:00100000:0.0:1788845033.772535:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1071 00000100:00100000:0.0:1788845033.772545:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641589632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.772571:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.772590:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287ad3ac lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b82 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.772678:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad3b3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.772694:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad3b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.772846:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad3b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845033.772877:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1445258 00000004:00200000:0.0:1788845033.772910:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845033.772933:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad3b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.772959:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287ad3ac lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b82 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.772976:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad3b3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b82 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.772984:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.772987:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845033.772989:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772a00, rc 0) 00000100:00100000:0.0:1788845033.773050:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641589632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 506us (737us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.773058:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1071 00000100:00100000:2.0:1788845033.776480:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.776521:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.776603:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641589888 00000100:00100000:3.0:1788845033.776626:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641589888 00000100:00100000:3.0:1788845033.776682:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 734 00000100:00100000:3.0:1788845033.776692:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641589888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.776720:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.776739:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad3ba lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b90 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.776795:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660279296:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.776928:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.777341:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641589760 00000100:00100000:0.0:1788845033.777347:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641589760 00000100:00100000:0.0:1788845033.777382:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1072 00000100:00100000:0.0:1788845033.777389:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641589760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.777401:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.777415:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad3c1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b89 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.777457:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.777483:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660279424:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845033.777529:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788845033.780567:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845033 00002000:00010000:3.0:1788845033.780579:0:6682:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 15 -> 153 00000100:00100000:3.0:1788845033.780609:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660279296:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.780647:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad3ba lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b90 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.780671:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad3ba lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b90 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.780684:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:3.0:1788845033.780750:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641589888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4059us (4228us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.780759:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 734 00000100:00100000:0.0:1788845033.781344:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660279424:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845033.781362:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845033.781376:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad3c1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b89 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.781390:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.781393:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.781398:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad3c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b89 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.781409:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773400, rc 0) 00000100:00100000:0.0:1788845033.781450:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641589760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4061us (4980us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.781459:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1072 00000100:00100000:2.0:1788845033.785521:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.785570:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.785687:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641590016 00000100:00100000:3.0:1788845033.785698:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641590016 00000100:00100000:3.0:1788845033.785769:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1073 00000100:00100000:3.0:1788845033.785779:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641590016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.785820:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.785850:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3c8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b97 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.785930:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.785974:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660279552:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.786046:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.788171:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660279552:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845033.788198:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845033.788216:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3c8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b97 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.788237:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.788240:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.788245:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3c8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b97 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.788273:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e800, rc 0) 00000100:00100000:3.0:1788845033.788373:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641590016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2595us (2862us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.788381:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1073 00000100:00100000:1.0:1788845033.789053:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641590144 00000100:00100000:1.0:1788845033.789060:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641590144 00000100:00100000:1.0:1788845033.789095:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 735 00000100:00100000:1.0:1788845033.789101:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641590144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.789113:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.789126:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3cf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8b9e expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.789184:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660279680:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845033.789227:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.794248:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660279680:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845033.794425:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3cf lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b9e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.794453:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3cf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b9e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.794466:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845033.794560:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641590144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5456us (8988us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845033.794570:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 735 00000100:00100000:2.0:1788845033.801087:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.801161:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641590272 00000100:00100000:3.0:1788845033.801171:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641590272 00000100:00100000:3.0:1788845033.801232:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1074 00000100:00100000:3.0:1788845033.801242:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641590272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.801280:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.801301:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad3d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ba5 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.801404:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.801420:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.801433:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845033.801460:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1445258 00000004:00200000:3.0:1788845033.801464:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845033.801482:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.801506:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad3d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.801524:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.801533:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.801535:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845033.801538:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fc00, rc 0) 00000100:00100000:3.0:1788845033.801600:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641590272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 359us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.801607:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1074 00000100:00100000:2.0:1788845033.806015:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.806180:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641590400 00000100:00100000:1.0:1788845033.806190:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641590400 00000100:00100000:1.0:1788845033.806262:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 736 00000100:00100000:1.0:1788845033.806273:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641590400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.806292:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.806311:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bac expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.806342:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.806377:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(5 11) 00010000:00010000:1.0:1788845033.806382:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:1.0:1788845033.806387:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1417216->1445887] (req 1417216->1445887) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.806426:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.806445:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 5 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.806460:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660279808:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845033.806551:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f840 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.809396:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.809458:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641590528 00000100:00100000:0.0:1788845033.809466:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641590528 00010000:00010000:0.0:1788845033.809498:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 6 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.809530:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 384 00000100:00100000:0.0:1788845033.809537:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641590528:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845033.809560:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845033.809580:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 6 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.809595:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 6 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.809618:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:0.0:1788845033.809621:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:0.0:1788845033.809627:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.809635:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.809661:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.809670:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660279936:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845033.809735:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788845033.810197:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287ad320 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b04 expref: 6 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.810208:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660279808:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845033.810225:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.810233:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845033.810264:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641590400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3993us (4261us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845033.810277:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 736 00000100:00100000:3.0:1788845033.814479:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.814560:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641590656 00000100:00100000:1.0:1788845033.814570:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641590656 00000100:00100000:1.0:1788845033.814640:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 384 00000100:00100000:1.0:1788845033.814650:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641590656:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:1.0:1788845033.815059:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1445258 -> 1445258 00002000:00010000:1.0:1788845033.815064:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00002000:00010000:1.0:1788845033.815066:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845033 -> 1788845029 00002000:00010000:1.0:1788845033.815069:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00002000:00010000:1.0:1788845033.815070:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 153 -> 153 00000100:00100000:1.0:1788845033.815130:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641590656:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 479us (656us total) trans 4294967429 rc 0/0 00000100:00100000:1.0:1788845033.815137:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 384 00000100:00100000:0.0:1788845033.815307:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660279936:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845033.815334:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845033.815358:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641590528:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5820us (5969us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.815366:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 384 00000100:00100000:0.0:1788845033.819592:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.820232:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641590784 00000100:00100000:2.0:1788845033.820243:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641590784 00000100:00100000:2.0:1788845033.820325:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1075 00000100:00100000:2.0:1788845033.820336:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641590784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845033.820369:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845033.820394:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ad3eb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bb3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.820438:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.820481:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660280064:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845033.820617:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845033.823502:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660280064:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845033.823532:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845033.823553:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ad3eb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bb3 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.823574:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.823577:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.823582:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287ad3eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bb3 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.823599:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:0.0:1788845033.823693:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641590784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3358us (4111us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845033.823702:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1075 00000100:00100000:0.0:1788845033.827516:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.827597:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641590912 00000100:00100000:3.0:1788845033.827609:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641590912 00000100:00100000:3.0:1788845033.827691:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1076 00000100:00100000:3.0:1788845033.827702:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641590912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.827731:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.827750:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad3f2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bba expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.827795:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.827850:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660280192:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.827957:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.830749:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660280192:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845033.830773:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845033.830993:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad3f2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bba expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.831017:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.831019:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845033.831024:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad3f2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bba expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.831040:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:2.0:1788845033.831342:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641590912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3639us (3832us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845033.831355:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1076 00000100:00100000:2.0:1788845033.834538:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.834639:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641591040 00000100:00100000:3.0:1788845033.834650:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641591040 00000100:00100000:3.0:1788845033.834755:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1077 00000100:00100000:3.0:1788845033.834783:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641591040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.834810:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.834842:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bc1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.834876:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.834919:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.834959:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.834978:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 22 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.834992:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660280320:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845033.835089:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.840321:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.840612:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660280320:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845033.840647:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.840669:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.840672:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845033.840677:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.840688:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e800, rc 0) 00000100:00100000:2.0:1788845033.840765:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641591040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5984us (6240us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845033.840775:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1077 00000100:00100000:1.0:1788845033.843861:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641591168 00000100:00100000:1.0:1788845033.843876:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641591168 00010000:00010000:1.0:1788845033.843938:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 23 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.844037:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 385 00000100:00100000:1.0:1788845033.844050:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641591168:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845033.844064:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845033.844093:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 23 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.844111:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 23 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.844159:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad39e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b7b expref: 23 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.844179:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845033.844186:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.844198:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.844209:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.844265:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.844279:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660280448:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845033.844382:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.851656:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.852151:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641591424 00000100:00100000:1.0:1788845033.852160:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641591424 00002000:00100000:1.0:1788845033.852215:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1445887) req@ffff949f03444e00 x1875739641591424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.852227:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03444e00 with x1875739641591424 ext(1048576->1445887) 00002000:00010000:1.0:1788845033.852233:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03444e00 00000100:00100000:1.0:1788845033.852281:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 385 00000100:00100000:1.0:1788845033.852290:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641591424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845033.853573:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660280448:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845033.853604:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845033.855542:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+22:2370:x1875739641591168:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11484us (15281us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.855560:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 385 00000100:00100000:0.0:1788845033.855772:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.858309:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641591296 00000100:00100000:3.0:1788845033.858336:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641591296 00002000:00100000:3.0:1788845033.858408:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f0000 x1875739641591296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.858421:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f0000 with x1875739641591296 00000100:00100000:3.0:1788845033.858475:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 711 00000100:00100000:3.0:1788845033.858486:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641591296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.866463:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641591424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14163us (14806us total) trans 0 rc 397312/397312 00000100:00100000:0.0:1788845033.866477:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 385 00002000:00100000:0.0:1788845033.866491:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1445887) req@ffff949f03444e00 x1875739641591424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 397312/397312 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.866502:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03444e00 with x1875739641591424 ext(1048576->1445887) 00002000:00010000:0.0:1788845033.866509:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03444e00 00000100:00100000:3.0:1788845033.876070:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641591296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17576us (20289us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1788845033.876085:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 711 00002000:00100000:3.0:1788845033.876098:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f0000 x1875739641591296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:139/0 lens 488/440 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845033.876109:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f0000 with x1875739641591296 00000100:00100000:2.0:1788845033.879759:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.879869:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641591552 00000100:00100000:0.0:1788845033.879880:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641591552 00000100:00100000:0.0:1788845033.879992:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1078 00000100:00100000:0.0:1788845033.880003:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:2370:x1875739641591552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845033.880025:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845033.880045:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287ad400 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bc8 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.880078:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287ad400 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc8 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845033.880090:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845033.880098:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287ad400 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc8 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.880111:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.880114:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.880118:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773000, rc 0) 00000100:00100000:0.0:1788845033.880165:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641591552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 164us (417us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.880172:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1078 00000100:00100000:0.0:1788845033.881801:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.881938:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641591680 00000100:00100000:3.0:1788845033.881948:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641591680 00000100:00100000:3.0:1788845033.882001:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 737 00000100:00100000:3.0:1788845033.882010:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641591680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.882028:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.882044:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad407 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bcf expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.882104:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660280576:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.882193:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c9c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788845033.885203:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845033 00000100:00100000:3.0:1788845033.885237:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660280576:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.885276:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad407 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bcf expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.885320:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad407 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bcf expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.885335:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:3.0:1788845033.885395:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641591680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3385us (3602us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.885404:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 737 00000100:00100000:2.0:1788845033.888636:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.888727:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641591808 00000100:00100000:3.0:1788845033.888737:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641591808 00000100:00100000:3.0:1788845033.888795:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 738 00000100:00100000:3.0:1788845033.888806:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641591808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845033.888826:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845033.888854:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad40e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bd6 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.888944:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660280704:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845033.889026:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845033.891765:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660280704:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845033.891805:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad40e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bd6 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.891842:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad40e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bd6 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.891875:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845033.891968:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641591808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3163us (3342us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845033.891976:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 738 00000100:00100000:2.0:1788845033.895216:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.895306:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641591936 00000100:00100000:1.0:1788845033.895315:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641591936 00000100:00100000:1.0:1788845033.895372:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 739 00000100:00100000:1.0:1788845033.895381:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641591936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845033.895398:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845033.895415:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad415 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bdd expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845033.895469:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660280832:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845033.895545:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845033.899150:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660280832:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845033.899202:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad415 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bdd expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.899231:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad415 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bdd expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845033.899245:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845033.899310:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641591936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3929us (4102us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845033.899321:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 739 00000100:00100000:0.0:1788845033.905093:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.905396:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641592064 00000100:00100000:2.0:1788845033.905408:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641592064 00000100:00100000:2.0:1788845033.905477:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1079 00000100:00100000:2.0:1788845033.905488:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641592064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845033.905511:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845033.905514:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845033.905561:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287ad400 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc8 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.905578:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845033.905583:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845033.905597:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8be4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.905621:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.905630:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.905665:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845033.905682:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 14 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845033.905696:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358f100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660280960:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845033.906042:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845033.912221:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845033.912325:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641592192 00000100:00100000:3.0:1788845033.912334:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641592192 00010000:00010000:3.0:1788845033.912381:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 15 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.912430:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 386 00000100:00100000:3.0:1788845033.912440:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641592192:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845033.912456:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845033.912485:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 15 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.912504:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 15 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.912525:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845033.912531:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.912544:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.912573:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845033.912611:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845033.912624:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660281088:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845033.912717:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c9c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788845033.913783:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad3f9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bc1 expref: 15 pid: 5840 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845033.913807:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358f100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660280960:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845033.913848:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845033.913880:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845033.914022:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845033.914026:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba200, rc 0) 00000100:00100000:0.0:1788845033.914085:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641592064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8598us (9001us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845033.914094:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1079 00000100:00100000:2.0:1788845033.918284:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.918360:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641592320 00000100:00100000:0.0:1788845033.918369:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641592320 00002000:00100000:0.0:1788845033.918420:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ca80 x1875739641592320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.918430:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ca80 with x1875739641592320 00000100:00100000:0.0:1788845033.918468:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 712 00000100:00100000:0.0:1788845033.918479:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641592320:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.918687:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641592320:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 209us (413us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.918695:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 712 00002000:00100000:0.0:1788845033.918702:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ca80 x1875739641592320/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.918718:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ca80 with x1875739641592320 00000100:00100000:3.0:1788845033.919546:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660281088:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845033.919570:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845033.919613:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641592192:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7174us (7400us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845033.919622:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 386 00000100:00100000:2.0:1788845033.927158:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.927269:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641592448 00000100:00100000:0.0:1788845033.927280:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641592448 00002000:00100000:0.0:1788845033.927340:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358df80 x1875739641592448/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.927362:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358df80 with x1875739641592448 00000100:00100000:0.0:1788845033.927417:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 713 00000100:00100000:0.0:1788845033.927428:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641592448:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.927638:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641592448:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 211us (488us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.927646:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 713 00002000:00100000:0.0:1788845033.927653:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358df80 x1875739641592448/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.927660:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358df80 with x1875739641592448 00000100:00100000:2.0:1788845033.931568:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.931859:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641592576 00000100:00100000:0.0:1788845033.931870:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641592576 00002000:00100000:0.0:1788845033.931954:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358dc00 x1875739641592576/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.931965:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358dc00 with x1875739641592576 00000100:00100000:0.0:1788845033.931993:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 714 00000100:00100000:0.0:1788845033.932003:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641592576:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.932237:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641592576:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (676us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.932245:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 714 00002000:00100000:0.0:1788845033.932252:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358dc00 x1875739641592576/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.932260:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358dc00 with x1875739641592576 00000100:00100000:2.0:1788845033.935957:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.936189:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641592704 00000100:00100000:0.0:1788845033.936199:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641592704 00002000:00100000:0.0:1788845033.936249:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358f100 x1875739641592704/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.936276:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358f100 with x1875739641592704 00000100:00100000:0.0:1788845033.936318:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 715 00000100:00100000:0.0:1788845033.936328:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641592704:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.936547:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641592704:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 220us (602us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.936555:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 715 00002000:00100000:0.0:1788845033.936562:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358f100 x1875739641592704/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.936570:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358f100 with x1875739641592704 00000100:00100000:2.0:1788845033.940415:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.940519:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641592832 00000100:00100000:0.0:1788845033.940531:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641592832 00002000:00100000:0.0:1788845033.940588:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ea00 x1875739641592832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.940600:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ea00 with x1875739641592832 00000100:00100000:0.0:1788845033.940642:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 716 00000100:00100000:0.0:1788845033.940653:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641592832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.940929:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641592832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (523us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.940937:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 716 00002000:00100000:0.0:1788845033.940944:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ea00 x1875739641592832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.940951:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ea00 with x1875739641592832 00000100:00100000:2.0:1788845033.944340:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.944496:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641592960 00000100:00100000:0.0:1788845033.944521:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641592960 00002000:00100000:0.0:1788845033.944569:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410e00 x1875739641592960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.944581:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410e00 with x1875739641592960 00000100:00100000:0.0:1788845033.944659:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 717 00000100:00100000:0.0:1788845033.944670:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641592960:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.945068:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641592960:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 398us (736us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.945077:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 717 00002000:00100000:0.0:1788845033.945085:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410e00 x1875739641592960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.945093:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410e00 with x1875739641592960 00000100:00100000:2.0:1788845033.948599:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.948695:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641593088 00000100:00100000:0.0:1788845033.948762:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641593088 00002000:00100000:0.0:1788845033.948815:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03412a00 x1875739641593088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.948837:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03412a00 with x1875739641593088 00000100:00100000:0.0:1788845033.948943:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 718 00000100:00100000:0.0:1788845033.948955:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641593088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.949162:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641593088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 209us (571us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.949169:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 718 00002000:00100000:0.0:1788845033.949176:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03412a00 x1875739641593088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.949202:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03412a00 with x1875739641593088 00000100:00100000:2.0:1788845033.952220:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.952525:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641593216 00000100:00100000:1.0:1788845033.952536:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641593216 00002000:00100000:1.0:1788845033.952598:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413100 x1875739641593216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.952609:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413100 with x1875739641593216 00000100:00100000:1.0:1788845033.952646:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 719 00000100:00100000:1.0:1788845033.952655:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641593216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845033.952875:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641593216:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 220us (662us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845033.952898:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 719 00002000:00100000:1.0:1788845033.952905:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413100 x1875739641593216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.952913:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413100 with x1875739641593216 00000100:00100000:0.0:1788845033.956649:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.956748:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641593344 00000100:00100000:2.0:1788845033.956759:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641593344 00002000:00100000:2.0:1788845033.956811:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f1c00 x1875739641593344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.956822:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1c00 with x1875739641593344 00000100:00100000:2.0:1788845033.956865:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 720 00000100:00100000:2.0:1788845033.956874:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641593344:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845033.957338:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641593344:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 462us (697us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845033.957356:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 720 00002000:00100000:2.0:1788845033.957364:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f1c00 x1875739641593344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.957372:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1c00 with x1875739641593344 00000100:00100000:2.0:1788845033.960430:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.960525:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641593472 00000100:00100000:0.0:1788845033.960535:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641593472 00002000:00100000:0.0:1788845033.960580:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413800 x1875739641593472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.960592:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413800 with x1875739641593472 00000100:00100000:0.0:1788845033.960665:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 721 00000100:00100000:0.0:1788845033.960675:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641593472:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845033.960967:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641593472:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (546us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845033.960976:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 721 00002000:00100000:0.0:1788845033.960984:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413800 x1875739641593472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845033.960992:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413800 with x1875739641593472 00000100:00100000:2.0:1788845033.965226:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845033.965454:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641593600 00000100:00100000:1.0:1788845033.965464:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641593600 00002000:00100000:1.0:1788845033.965522:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03411500 x1875739641593600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.965535:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03411500 with x1875739641593600 00000100:00100000:1.0:1788845033.965613:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 722 00000100:00100000:1.0:1788845033.965624:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641593600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845033.966000:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641593600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 377us (785us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845033.966009:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 722 00002000:00100000:1.0:1788845033.966017:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03411500 x1875739641593600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/4536 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845033.966025:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03411500 with x1875739641593600 00000100:00100000:1.0:1788845033.969125:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845033.969228:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641593728 00000100:00100000:2.0:1788845033.969238:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641593728 00002000:00100000:2.0:1788845033.969283:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03447480 x1875739641593728/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/0 e 0 to 0 dl 1788845044 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.969295:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03447480 with x1875739641593728 00000100:00100000:2.0:1788845033.969332:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 723 00000100:00100000:2.0:1788845033.969341:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641593728:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845033.973285:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788845033.973298:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845033 00000004:00010000:2.0:1788845033.973301:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:2.0:1788845033.973304:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845033 00000004:00010000:2.0:1788845033.973307:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:2.0:1788845033.973535:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2370:x1875739641593728:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4189us (4414us total) trans 4294967702 rc 0/0 00000100:00100000:2.0:1788845033.973546:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 723 00002000:00100000:2.0:1788845033.973559:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03447480 x1875739641593728/t4294967702(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:139/0 lens 488/448 e 0 to 0 dl 1788845044 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845033.973570:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03447480 with x1875739641593728 00000100:00100000:2.0:1788845033.976216:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845033.976282:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641593856 00000100:00100000:0.0:1788845033.976291:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641593856 00000100:00100000:0.0:1788845033.976362:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1080 00000100:00100000:0.0:1788845033.976372:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641593856:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845034.016965:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967694 is committed 00000001:00080000:1.0:1788845034.016985:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967691 is committed 00000001:00080000:1.0:1788845034.016992:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967699 is committed 00000001:00080000:1.0:1788845034.016998:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967702 is committed 00000001:00080000:1.0:1788845034.017005:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967695 is committed 00000001:00080000:1.0:1788845034.017014:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967698 is committed 00000100:00100000:2.0:1788845034.017317:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641593856:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 40939us (41104us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.017330:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1080 00000100:00100000:1.0:1788845034.020558:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.021942:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641593984 00000100:00100000:3.0:1788845034.021957:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641593984 00000100:00100000:2.0:1788845034.022221:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1081 00000100:00100000:2.0:1788845034.022234:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641593984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.022256:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.022282:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad423 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8beb expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.022326:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.022389:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660281216:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.022544:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.026150:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660281216:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.026180:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845034.026203:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad423 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8beb expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.026248:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.026252:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.026258:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad423 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8beb expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.026274:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba800, rc 0) 00000100:00100000:0.0:1788845034.026336:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641593984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4104us (5785us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.026411:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1081 00000100:00100000:2.0:1788845034.029412:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.029574:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641594112 00000100:00100000:3.0:1788845034.029584:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641594112 00000100:00100000:3.0:1788845034.029637:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1082 00000100:00100000:3.0:1788845034.029647:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641594112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.029666:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.029684:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad42a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bf2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.029729:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.029798:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660281344:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.029965:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.033448:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660281344:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845034.033479:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845034.033502:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad42a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bf2 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.033525:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.033527:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845034.033532:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad42a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bf2 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.033546:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845034.033669:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641594112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4021us (4267us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.033680:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1082 00000100:00100000:1.0:1788845034.038054:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.038153:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641594240 00000100:00100000:0.0:1788845034.038165:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641594240 00000100:00100000:0.0:1788845034.038231:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1083 00000100:00100000:0.0:1788845034.038242:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641594240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.038434:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.038460:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287ad431 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8bf9 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.038501:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.038542:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660281472:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.038672:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.043131:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660281472:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845034.043158:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845034.043178:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287ad431 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bf9 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.043199:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.043201:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845034.043206:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287ad431 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bf9 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.043240:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773a00, rc 0) 00000100:00100000:3.0:1788845034.043299:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641594240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5057us (5259us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.043309:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1083 00000100:00100000:1.0:1788845034.047683:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.048050:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641594368 00000100:00100000:2.0:1788845034.048059:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641594368 00000100:00100000:2.0:1788845034.048147:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1084 00000100:00100000:2.0:1788845034.048158:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641594368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.048180:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.048202:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287ad438 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c00 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.048240:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.048293:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660281600:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.048397:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d66c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.052618:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660281600:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.052648:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788845034.052672:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287ad438 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c00 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.052780:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.052784:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.052789:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287ad438 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c00 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.052808:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb400, rc 0) 00000100:00100000:0.0:1788845034.052914:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641594368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4728us (5211us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.052924:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1084 00000100:00100000:0.0:1788845034.057162:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.057387:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641594624 00000100:00100000:2.0:1788845034.057397:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641594624 00000100:00100000:2.0:1788845034.057457:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1085 00000100:00100000:2.0:1788845034.057467:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641594624:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845034.057524:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.057556:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad3b3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8b82 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.057576:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.057581:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845034.058188:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.058206:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.058215:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.058643:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.058666:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 14 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.058681:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660281728:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845034.058838:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.062298:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660281728:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845034.062330:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.064549:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.064713:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641594752 00000100:00100000:2.0:1788845034.064724:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641594752 00010000:00010000:2.0:1788845034.064772:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 15 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.064810:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 387 00000100:00100000:2.0:1788845034.064821:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641594752:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845034.064848:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.064855:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 15 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.064872:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 15 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.064926:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad3dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ba5 expref: 15 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.064942:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.064948:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.064957:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.064964:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.064985:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845034.065046:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641594752:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 226us (506us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.065054:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 387 00010000:00010000:0.0:1788845034.065163:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.065171:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845034.065312:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967703 00000004:00010000:0.0:1788845034.065378:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788845034.065382:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845033 -> 1788845034 00000004:00010000:0.0:1788845034.065385:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:0.0:1788845034.065387:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845033 -> 1788845034 00000004:00010000:0.0:1788845034.065405:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 454 00000004:00010000:0.0:1788845034.065414:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005e8f993b reply state 000000006a9c2c9c transno 4294967704 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.065424:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845034.065493:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641594624:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8027us (8342us total) trans 4294967704 rc 0/0 00000100:00100000:0.0:1788845034.065500:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1085 00000020:00080000:0.0:1788845034.065504:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788845034.068421:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845034.068962:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.068975:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.068982:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.068988:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008cba400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845034.068993:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.069000:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.069019:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.069021:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845034.069025:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.069032:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad43f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.069129:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641594880 00000100:00100000:0.0:1788845034.069138:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641594880 00000100:00100000:0.0:1788845034.069202:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1086 00000100:00100000:0.0:1788845034.069212:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641594880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.069237:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.069264:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c07 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.069294:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.069303:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.069337:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.069366:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 13 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.069403:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660281856:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845034.069488:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00010000:00010000:2.0:1788845034.069756:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845034.072661:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.072737:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641595008 00000100:00100000:2.0:1788845034.072747:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641595008 00010000:00010000:2.0:1788845034.072786:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 14 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.072839:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 388 00000100:00100000:2.0:1788845034.072849:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641595008:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845034.072868:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.072924:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 14 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.072941:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 14 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.072961:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.072967:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.072978:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.072987:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.073026:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.073038:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660281984:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845034.073114:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788845034.074265:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba200/0xb2187301287ad41c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8be4 expref: 14 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.074290:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660281856:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845034.074318:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.074331:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.074335:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.074337:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773200, rc 0) 00000100:00100000:0.0:1788845034.074405:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641594880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5194us (5994us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.074414:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1086 00000100:00100000:2.0:1788845034.077127:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660281984:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845034.077159:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845034.077215:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641595008:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4366us (4562us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.077228:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 388 00000100:00100000:2.0:1788845034.079923:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.079963:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.080025:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641595264 00000100:00100000:3.0:1788845034.080043:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641595264 00002000:00010000:3.0:1788845034.080080:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845034.080083:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03411c00 with x1875739641595264 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845034.080093:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03411c00. 00000100:00100000:3.0:1788845034.080119:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 386 00000100:00100000:3.0:1788845034.080126:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641595264:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845034.080425:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1445258 -> 0 00002000:00010000:3.0:1788845034.080428:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845033 -> 1788845034 00002000:00010000:3.0:1788845034.080431:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845033 -> 1788845029 00002000:00010000:3.0:1788845034.080432:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845033 -> 1788845034 00002000:00010000:3.0:1788845034.080434:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 153 -> 153 00000100:00100000:3.0:1788845034.080481:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641595264:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 356us (568us total) trans 4294967430 rc 0/0 00000100:00100000:3.0:1788845034.080486:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 386 00002000:00010000:3.0:1788845034.080489:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845034.080490:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03411c00 with x1875739641595264 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845034.080494:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03411c00. 00000100:00100000:1.0:1788845034.080910:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641595136 00000100:00100000:1.0:1788845034.080915:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641595136 00002000:00010000:1.0:1788845034.080936:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845034.080940:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ca80 with x1875739641595136 00000100:00100000:1.0:1788845034.080960:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 724 00000100:00100000:1.0:1788845034.080964:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641595136:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845034.081112:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 357427 00000004:00010000:1.0:1788845034.081114:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:1.0:1788845034.081117:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:1.0:1788845034.081118:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:1.0:1788845034.081120:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 454 -> 454 00000100:00100000:1.0:1788845034.081155:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641595136:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 190us (1190us total) trans 4294967705 rc 0/0 00000100:00100000:1.0:1788845034.081159:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 724 00002000:00010000:1.0:1788845034.081161:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845034.081170:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ca80 with x1875739641595136 00000100:00100000:2.0:1788845034.087218:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.087545:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641595392 00000100:00100000:3.0:1788845034.087555:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641595392 00000100:00100000:3.0:1788845034.087648:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1087 00000100:00100000:3.0:1788845034.087659:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641595392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.087680:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.087767:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad44d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c0e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.087996:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.088027:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.088039:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845034.088069:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 357427 00000004:00200000:3.0:1788845034.088073:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845034.088093:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.088119:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad44d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.088135:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.088144:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.088147:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845034.088151:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:3.0:1788845034.088267:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641595392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 598us (1047us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845034.088274:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1087 00000100:00100000:0.0:1788845034.090759:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.090836:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.091015:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641595648 00000100:00100000:2.0:1788845034.091026:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641595648 00000100:00100000:2.0:1788845034.091078:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 740 00000100:00100000:2.0:1788845034.091089:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641595648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.091110:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.091132:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad45b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c1c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.091209:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660282112:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.091311:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.093333:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641595520 00000100:00100000:3.0:1788845034.093403:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641595520 00002000:00010000:3.0:1788845034.093880:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845033 00000100:00100000:3.0:1788845034.093938:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660282112:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845034.093976:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad45b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c1c expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.094024:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad45b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c1c expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.094040:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbac00, rc 0) 00000100:00100000:3.0:1788845034.094094:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641595648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3007us (3257us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.094102:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 740 00000100:00100000:2.0:1788845034.095654:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1088 00000100:00100000:2.0:1788845034.095667:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641595520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.095687:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.095717:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad462 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c15 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.095753:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.095803:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660282240:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.095916:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d60c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.098674:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660282240:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845034.098704:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 357427 00010000:00010000:2.0:1788845034.098723:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad462 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c15 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.098743:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.098746:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845034.098751:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad462 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c15 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.098769:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba800, rc 0) 00000100:00100000:2.0:1788845034.098840:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641595520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3175us (8087us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.098849:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1088 00000100:00100000:3.0:1788845034.101940:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.101976:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.102061:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641595904 00000100:00100000:1.0:1788845034.102074:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641595904 00000100:00100000:1.0:1788845034.102138:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 741 00000100:00100000:1.0:1788845034.102149:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641595904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.102169:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.102189:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad469 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c2a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.102251:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5df800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660282368:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.102366:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.102606:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641595776 00000100:00100000:0.0:1788845034.102612:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641595776 00000100:00100000:0.0:1788845034.102658:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1089 00000100:00100000:0.0:1788845034.102667:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641595776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.102679:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.102695:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad470 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c23 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.102753:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.102780:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660282496:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.102819:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.105770:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5df800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660282368:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845034.105814:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad469 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c2a expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.105852:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad469 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c2a expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.105866:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845034.105983:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641595904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3836us (4005us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.105992:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 741 00000100:00100000:2.0:1788845034.108192:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660282496:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845034.108218:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 357427 00010000:00010000:2.0:1788845034.108238:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad470 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c23 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.108273:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.108277:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845034.108281:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad470 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c23 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.108306:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773400, rc 0) 00000100:00100000:2.0:1788845034.108416:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641595776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5750us (6547us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.108425:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1089 00000100:00100000:1.0:1788845034.112135:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.112218:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.112294:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641596160 00000100:00100000:3.0:1788845034.112305:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641596160 00000100:00100000:3.0:1788845034.112372:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 742 00000100:00100000:3.0:1788845034.112382:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641596160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.112403:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.112422:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad477 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c38 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.112482:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660282624:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.112572:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.114986:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660282624:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845034.115028:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad477 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c38 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.115077:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad477 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c38 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.115090:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845034.115152:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641596160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2771us (2933us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.115161:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 742 00000100:00100000:0.0:1788845034.116079:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641596032 00000100:00100000:0.0:1788845034.116090:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641596032 00000100:00100000:0.0:1788845034.116187:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1090 00000100:00100000:0.0:1788845034.116197:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641596032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.116222:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.116244:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad47e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c31 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.116295:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.116334:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360eca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660282752:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.116518:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.120130:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360eca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660282752:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.120153:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 357427 00010000:00010000:0.0:1788845034.120169:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad47e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c31 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.120187:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.120189:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.120206:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad47e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c31 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.120217:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772a00, rc 0) 00000100:00100000:0.0:1788845034.120273:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641596032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4078us (8151us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.120280:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1090 00000100:00100000:2.0:1788845034.123162:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.123194:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.123261:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641596416 00000100:00100000:3.0:1788845034.123269:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641596416 00000100:00100000:3.0:1788845034.123318:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 743 00000100:00100000:3.0:1788845034.123325:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641596416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.123347:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.123362:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad485 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c46 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.123411:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660282880:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.123475:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.123650:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641596288 00000100:00100000:0.0:1788845034.123653:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641596288 00000100:00100000:0.0:1788845034.123676:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1091 00000100:00100000:0.0:1788845034.123680:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641596288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.123688:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.123704:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad48c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c3f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.123728:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.123745:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360eca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660283008:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.123778:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.126591:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360eca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660283008:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.126619:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 357427 00010000:00010000:0.0:1788845034.126638:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad48c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c3f expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.126658:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.126662:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.126682:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad48c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c3f expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.126696:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773e00, rc 0) 00000100:00100000:0.0:1788845034.126759:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641596288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3077us (3604us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.126768:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1091 00000100:00100000:3.0:1788845034.126876:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660282880:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845034.126932:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad485 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c46 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.126952:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad485 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c46 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.126975:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845034.127000:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641596416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3675us (3804us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.127007:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 743 00000100:00100000:3.0:1788845034.130750:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.130842:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641596544 00000100:00100000:0.0:1788845034.130852:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641596544 00000100:00100000:0.0:1788845034.130967:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 725 00000100:00100000:0.0:1788845034.130978:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641596544:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845034.131227:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 357427 -> 357427 00000004:00010000:0.0:1788845034.131231:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.131234:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:0.0:1788845034.131236:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.131239:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 454 -> 454 00000100:00100000:0.0:1788845034.131307:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641596544:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 330us (566us total) trans 4294967706 rc 0/0 00000100:00100000:0.0:1788845034.131315:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 725 00000100:00100000:2.0:1788845034.135082:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.135178:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641596672 00000100:00100000:1.0:1788845034.135188:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641596672 00000100:00100000:1.0:1788845034.135263:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1092 00000100:00100000:1.0:1788845034.135274:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641596672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.135299:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.135320:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ad493 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c4d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.135418:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad49a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.135435:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad49a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.135447:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad49a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845034.135475:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 357427 00000004:00200000:1.0:1788845034.135479:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845034.135495:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad49a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.135521:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ad493 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c4d expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.135539:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad49a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c4d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.135548:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.135551:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845034.135554:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845034.135614:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641596672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 342us (541us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.135628:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1092 00000100:00100000:1.0:1788845034.139129:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.139216:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641596800 00000100:00100000:2.0:1788845034.139227:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641596800 00000100:00100000:2.0:1788845034.139294:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1093 00000100:00100000:2.0:1788845034.139305:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641596800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.139325:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.139358:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad4a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c54 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.139397:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.139439:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660283136:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.139526:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d60c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.141419:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.141644:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641596928 00000100:00100000:2.0:1788845034.141653:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641596928 00000100:00100000:2.0:1788845034.141711:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 744 00000100:00100000:2.0:1788845034.141722:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641596928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.141740:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.141757:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ad4a8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c5b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.141815:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660283264:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.141935:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.144608:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660283264:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.144651:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ad4a8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c5b expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.144676:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ad4a8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c5b expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.144693:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:2.0:1788845034.144760:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641596928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3038us (3347us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.144769:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 744 00000100:00100000:2.0:1788845034.147320:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660283136:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845034.147359:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 357427 00010000:00010000:2.0:1788845034.147380:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad4a1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c54 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.147400:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.147403:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845034.147408:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad4a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c54 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.147430:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbaa00, rc 0) 00000100:00100000:2.0:1788845034.147487:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641596800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8183us (8369us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.147496:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1093 00000100:00100000:3.0:1788845034.150682:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.150834:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641597056 00000100:00100000:1.0:1788845034.150844:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641597056 00000100:00100000:1.0:1788845034.151012:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1094 00000100:00100000:1.0:1788845034.151024:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641597056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.151046:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.151069:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ad4af lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c62 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.151106:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.151146:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dd180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660283392:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.151226:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.153360:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.153490:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641597184 00000100:00100000:1.0:1788845034.153501:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641597184 00000100:00100000:1.0:1788845034.153559:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 745 00000100:00100000:1.0:1788845034.153569:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641597184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.153589:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.153607:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad4b6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c69 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.153664:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660283520:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.153761:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.156513:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dd180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660283392:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.156544:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 357427 00010000:00010000:1.0:1788845034.156565:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ad4af lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c62 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.156586:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.156589:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.156594:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ad4af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c62 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.156610:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:1.0:1788845034.156673:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641597056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5651us (6003us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.156682:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1094 00000100:00100000:1.0:1788845034.159029:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660283520:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845034.159069:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad4b6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c69 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.159093:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad4b6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c69 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.159107:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:1.0:1788845034.159217:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641597184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5648us (5874us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.159225:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 745 00000100:00100000:2.0:1788845034.163216:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.163473:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641597440 00000100:00100000:0.0:1788845034.163483:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641597440 00000100:00100000:0.0:1788845034.163574:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1095 00000100:00100000:0.0:1788845034.163585:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641597440:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845034.163646:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845034.163673:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad49a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c4d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.163721:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.163726:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845034.163981:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.163997:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.164004:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.164036:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.164054:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 13 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.164067:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360eca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660283648:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845034.164147:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.166641:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.166727:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641597568 00000100:00100000:1.0:1788845034.166747:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641597568 00010000:00010000:1.0:1788845034.166782:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 14 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.166834:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 389 00000100:00100000:1.0:1788845034.166845:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641597568:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845034.166861:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845034.166868:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 14 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.166906:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 14 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.166928:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.166934:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.166942:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.166950:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.166969:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845034.167023:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641597568:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 180us (391us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.167032:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 389 00010000:00010000:0.0:1788845034.167335:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad454 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c0e expref: 13 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.167363:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360eca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660283648:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845034.167384:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.167391:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.167404:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845034.167517:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967707 00000004:00010000:0.0:1788845034.167570:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 357427 -> 357427 00000004:00010000:0.0:1788845034.167574:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.167576:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:0.0:1788845034.167578:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.167580:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 454 -> 454 00000004:00010000:0.0:1788845034.167587:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009ccd7b79 reply state 00000000a6278441 transno 4294967708 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.167595:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845034.167664:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641597440:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4081us (4459us total) trans 4294967708 rc 0/0 00000100:00100000:0.0:1788845034.167671:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1095 00000020:00080000:0.0:1788845034.167675:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845034.171013:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.171100:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641597696 00000100:00100000:1.0:1788845034.171110:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641597696 00002000:00010000:1.0:1788845034.171164:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1541764->18446744073709551615) 00002000:00100000:1.0:1788845034.171169:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01bb1c00 with x1875739641597696 ext(1541764->18446744073709551615) 00002000:00010000:1.0:1788845034.171180:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01bb1c00. 00000100:00100000:1.0:1788845034.171213:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 387 00000100:00100000:1.0:1788845034.171223:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641597696:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845034.171468:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1541764 00002000:00010000:1.0:1788845034.171472:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00002000:00010000:1.0:1788845034.171475:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845033 -> 1788845029 00002000:00010000:1.0:1788845034.171477:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00002000:00010000:1.0:1788845034.171480:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 153 -> 153 00000100:00100000:1.0:1788845034.171531:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641597696:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 309us (528us total) trans 4294967431 rc 0/0 00000100:00100000:1.0:1788845034.171539:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 387 00002000:00010000:1.0:1788845034.171543:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1541764->18446744073709551615) 00002000:00100000:1.0:1788845034.171546:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01bb1c00 with x1875739641597696 ext(1541764->18446744073709551615) 00002000:00010000:1.0:1788845034.171550:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01bb1c00. 00010000:00010000:2.0:1788845034.171773:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.171783:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.171791:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.171797:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845034.171802:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.171809:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.171834:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.171837:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845034.171842:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.171848:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad4bd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.175536:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.175609:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641597824 00000100:00100000:1.0:1788845034.175619:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641597824 00002000:00100000:1.0:1788845034.175669:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1544191) req@ffff949f0251c700 x1875739641597824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.175681:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251c700 with x1875739641597824 ext(1445888->1544191) 00002000:00010000:1.0:1788845034.175687:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251c700 00000100:00100000:1.0:1788845034.175730:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 388 00000100:00100000:1.0:1788845034.175739:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641597824:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.180821:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641597824:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 5071us (5282us total) trans 0 rc 98304/98304 00000100:00100000:1.0:1788845034.180862:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 388 00002000:00100000:1.0:1788845034.180877:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1544191) req@ffff949f0251c700 x1875739641597824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 98304/98304 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.180918:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251c700 with x1875739641597824 ext(1445888->1544191) 00002000:00010000:1.0:1788845034.180925:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251c700 00000100:00100000:3.0:1788845034.186304:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.186511:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641597952 00000100:00100000:0.0:1788845034.186521:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641597952 00002000:00100000:0.0:1788845034.186567:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1507328->1541763) req@ffff949f01bb0700 x1875739641597952/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.186579:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01bb0700 with x1875739641597952 ext(1507328->1541763) 00002000:00010000:0.0:1788845034.186584:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01bb0700 00000100:00100000:0.0:1788845034.186613:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 389 00000100:00100000:0.0:1788845034.186623:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641597952:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845034.190468:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641597952:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3837us (4164us total) trans 4294967432 rc 0/0 00000100:00100000:1.0:1788845034.190482:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 389 00002000:00100000:1.0:1788845034.190495:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1507328->1541763) req@ffff949f01bb0700 x1875739641597952/t4294967432(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/448 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.190505:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01bb0700 with x1875739641597952 ext(1507328->1541763) 00002000:00010000:1.0:1788845034.190511:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01bb0700 00000100:00100000:3.0:1788845034.192993:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.193086:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641598080 00000100:00100000:0.0:1788845034.193119:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641598080 00000100:00100000:0.0:1788845034.193179:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 746 00000100:00100000:0.0:1788845034.193189:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641598080:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:2.0:1788845034.218532:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641598080:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 25334us (25539us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.218547:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 746 00000001:00080000:1.0:1788845034.220939:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967432 is committed 00000100:00100000:0.0:1788845034.222237:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.222319:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641598208 00000100:00100000:3.0:1788845034.222329:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641598208 00000100:00100000:3.0:1788845034.222397:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1096 00000100:00100000:3.0:1788845034.222408:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641598208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.222434:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.222455:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad4c4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c70 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.222538:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.222553:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.222579:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845034.222607:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1541764 00000004:00200000:3.0:1788845034.222612:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845034.222630:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.222658:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287ad4c4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.222675:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.222683:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.222686:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845034.222690:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:3.0:1788845034.222761:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641598208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 355us (532us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845034.222769:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1096 00000100:00100000:2.0:1788845034.226924:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.227041:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641598336 00000100:00100000:1.0:1788845034.227051:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641598336 00002000:00100000:1.0:1788845034.227100:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1052671) req@ffff949f0251e300 x1875739641598336/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.227111:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251e300 with x1875739641598336 ext(1048576->1052671) 00002000:00010000:1.0:1788845034.227117:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251e300 00000100:00100000:1.0:1788845034.227150:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 390 00000100:00100000:1.0:1788845034.227160:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641598336:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.227389:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641598336:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 230us (518us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845034.227415:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 390 00002000:00100000:1.0:1788845034.227423:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1052671) req@ffff949f0251e300 x1875739641598336/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.227430:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251e300 with x1875739641598336 ext(1048576->1052671) 00002000:00010000:1.0:1788845034.227433:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251e300 00000100:00100000:0.0:1788845034.233051:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.233352:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641598464 00000100:00100000:1.0:1788845034.233362:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641598464 00002000:00100000:1.0:1788845034.233406:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1052672->1056767) req@ffff94a0360ec000 x1875739641598464/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.233418:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ec000 with x1875739641598464 ext(1052672->1056767) 00002000:00010000:1.0:1788845034.233423:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ec000 00000100:00100000:1.0:1788845034.233457:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 391 00000100:00100000:1.0:1788845034.233466:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641598464:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.233721:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641598464:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (680us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845034.233729:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 391 00002000:00100000:1.0:1788845034.233735:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1052672->1056767) req@ffff94a0360ec000 x1875739641598464/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.233744:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ec000 with x1875739641598464 ext(1052672->1056767) 00002000:00010000:1.0:1788845034.233747:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ec000 00000100:00100000:0.0:1788845034.237232:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.237325:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641598592 00000100:00100000:3.0:1788845034.237336:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641598592 00002000:00100000:3.0:1788845034.237447:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1056768->1060863) req@ffff94a0360eea00 x1875739641598592/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845034.237460:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360eea00 with x1875739641598592 ext(1056768->1060863) 00002000:00010000:3.0:1788845034.237467:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360eea00 00000100:00100000:3.0:1788845034.237545:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 392 00000100:00100000:3.0:1788845034.237569:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641598592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845034.238018:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641598592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 449us (795us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845034.238026:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 392 00002000:00100000:3.0:1788845034.238034:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1056768->1060863) req@ffff94a0360eea00 x1875739641598592/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845034.238042:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360eea00 with x1875739641598592 ext(1056768->1060863) 00002000:00010000:3.0:1788845034.238046:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360eea00 00000100:00100000:2.0:1788845034.241231:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.241402:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641598720 00000100:00100000:1.0:1788845034.241412:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641598720 00002000:00100000:1.0:1788845034.241465:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1060864->1064959) req@ffff949f0251f800 x1875739641598720/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.241496:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251f800 with x1875739641598720 ext(1060864->1064959) 00002000:00010000:1.0:1788845034.241503:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251f800 00000100:00100000:1.0:1788845034.241538:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 393 00000100:00100000:1.0:1788845034.241548:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641598720:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.242031:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641598720:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 482us (809us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845034.242040:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 393 00002000:00100000:1.0:1788845034.242048:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1060864->1064959) req@ffff949f0251f800 x1875739641598720/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.242056:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251f800 with x1875739641598720 ext(1060864->1064959) 00002000:00010000:1.0:1788845034.242060:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251f800 00000100:00100000:0.0:1788845034.245141:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.245247:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641598848 00000100:00100000:2.0:1788845034.245257:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641598848 00002000:00100000:2.0:1788845034.245303:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1064960->1069055) req@ffff94a0360efb80 x1875739641598848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.245315:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360efb80 with x1875739641598848 ext(1064960->1069055) 00002000:00010000:2.0:1788845034.245321:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360efb80 00000100:00100000:2.0:1788845034.245446:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 394 00000100:00100000:2.0:1788845034.245458:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641598848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.245691:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641598848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (559us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845034.245710:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 394 00002000:00100000:2.0:1788845034.245718:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1064960->1069055) req@ffff94a0360efb80 x1875739641598848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.245727:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360efb80 with x1875739641598848 ext(1064960->1069055) 00002000:00010000:2.0:1788845034.245731:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360efb80 00000100:00100000:0.0:1788845034.250156:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.250274:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641598976 00000100:00100000:2.0:1788845034.250285:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641598976 00000100:00100000:2.0:1788845034.250343:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1097 00000100:00100000:2.0:1788845034.250354:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641598976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.250375:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.250397:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbae00/0xb2187301287ad4d2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c77 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.250480:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad4d9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.250495:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad4d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.250507:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad4d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845034.250531:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1541764 00000004:00200000:2.0:1788845034.250535:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845034.250555:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad4d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.250580:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbae00/0xb2187301287ad4d2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c77 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.250598:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad4d9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c77 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.250606:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.250608:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845034.250611:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba800, rc 0) 00000100:00100000:2.0:1788845034.250762:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641598976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 410us (616us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.250769:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1097 00000100:00100000:3.0:1788845034.253715:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.253760:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.254102:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641599104 00000100:00100000:0.0:1788845034.254111:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641599104 00000100:00100000:0.0:1788845034.254172:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1098 00000100:00100000:0.0:1788845034.254182:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641599104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.254203:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.254225:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4e0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c7e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.254264:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.254306:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660283776:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.254399:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000004:00010000:0.0:1788845034.256298:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845033 -> 1788845034 00000100:00100000:0.0:1788845034.256326:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660283776:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.256346:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 357427 00010000:00010000:0.0:1788845034.256363:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4e0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c7e expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.256407:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.256410:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.256416:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c7e expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.256431:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772800, rc 0) 00000100:00100000:0.0:1788845034.256490:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641599104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2309us (2795us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.256499:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1098 00000100:00100000:1.0:1788845034.258035:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641599232 00000100:00100000:1.0:1788845034.258045:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641599232 00000100:00100000:1.0:1788845034.258106:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 747 00000100:00100000:1.0:1788845034.258115:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641599232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.258132:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.258151:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad4e7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c85 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.258205:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660283904:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.258371:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845034.261636:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845029 -> 1788845034 00000100:00100000:1.0:1788845034.261667:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660283904:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845034.261733:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad4e7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c85 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.261759:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad4e7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c85 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.261773:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:1.0:1788845034.261833:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641599232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3709us (8061us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.261842:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 747 00000100:00100000:2.0:1788845034.265674:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.265802:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.266422:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641599488 00000100:00100000:3.0:1788845034.266432:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641599488 00000100:00100000:3.0:1788845034.266490:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 748 00000100:00100000:3.0:1788845034.266500:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641599488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.266518:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.266538:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad4ee lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c93 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.266595:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660284032:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.266859:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.267034:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641599360 00000100:00100000:0.0:1788845034.267039:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641599360 00000100:00100000:0.0:1788845034.267176:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1099 00000100:00100000:0.0:1788845034.267185:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641599360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.267197:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.267209:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4f5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c8c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.267239:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.267409:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660284160:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.267450:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.270151:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660284160:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.270176:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 357427 00010000:00010000:0.0:1788845034.270195:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4f5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c8c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.270233:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.270236:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.270242:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c8c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.272485:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772800, rc 0) 00000100:00100000:0.0:1788845034.272588:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641599360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5401us (6922us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.272595:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1099 00000100:00100000:2.0:1788845034.273809:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660284032:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.273858:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad4ee lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c93 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.275636:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287ad4ee lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c93 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.275656:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:2.0:1788845034.275877:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641599488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9377us (10073us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.275916:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 748 00000100:00100000:2.0:1788845034.296140:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.296381:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641599616 00000100:00100000:1.0:1788845034.296392:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641599616 00002000:00100000:1.0:1788845034.296441:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251f100 x1875739641599616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.296451:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251f100 with x1875739641599616 00000100:00100000:1.0:1788845034.296544:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 726 00000100:00100000:1.0:1788845034.296555:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641599616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.302755:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.305185:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641599616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8620us (9045us total) trans 0 rc 360448/360448 00000100:00100000:1.0:1788845034.305200:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 726 00002000:00100000:1.0:1788845034.305214:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251f100 x1875739641599616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 360448/360448 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.305224:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251f100 with x1875739641599616 00000100:00100000:3.0:1788845034.312221:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.312336:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641599872 00000100:00100000:0.0:1788845034.312357:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641599872 00000100:00100000:0.0:1788845034.312422:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1100 00000100:00100000:0.0:1788845034.312433:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641599872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.312455:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.312475:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8c9a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.312504:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.312515:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.312547:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.312586:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 16 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.312601:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660284288:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845034.312673:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.314101:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641599744 00000100:00100000:2.0:1788845034.314113:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641599744 00002000:00100000:2.0:1788845034.314169:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1069056->1445887) req@ffff94a03d5de680 x1875739641599744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.314180:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5de680 with x1875739641599744 ext(1069056->1445887) 00002000:00010000:2.0:1788845034.314186:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5de680 00000100:00100000:2.0:1788845034.314209:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 395 00000100:00100000:2.0:1788845034.314218:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641599744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.318848:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660284288:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845034.318915:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.318936:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.318939:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.318944:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.318955:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772800, rc 0) 00000100:00100000:0.0:1788845034.319018:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641599872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6586us (6804us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.319027:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1100 00000100:00100000:2.0:1788845034.325508:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641599744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 11279us (22754us total) trans 0 rc 376832/376832 00000100:00100000:2.0:1788845034.325524:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 395 00002000:00100000:2.0:1788845034.325537:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1069056->1445887) req@ffff94a03d5de680 x1875739641599744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 376832/376832 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.325548:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5de680 with x1875739641599744 ext(1069056->1445887) 00002000:00010000:2.0:1788845034.325554:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5de680 00000100:00100000:0.0:1788845034.329925:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.332002:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641600000 00000100:00100000:2.0:1788845034.332015:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641600000 00010000:00010000:2.0:1788845034.332062:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 17 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.332159:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 390 00000100:00100000:2.0:1788845034.332170:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d4a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641600000:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845034.332185:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.332213:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 17 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.332232:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 17 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.332265:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287ad446 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c07 expref: 17 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.332285:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.332292:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.332304:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.332319:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.332368:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.332381:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660284416:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845034.332516:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.338804:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.338909:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641600128 00000100:00100000:0.0:1788845034.338918:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641600128 00000100:00100000:0.0:1788845034.338976:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1101 00000100:00100000:0.0:1788845034.338986:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2367:x1875739641600128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.339009:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.339046:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad503 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ca1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.339083:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.339123:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660284544:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.339206:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.339943:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660284416:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845034.339965:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845034.339997:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d4a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641600000:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7829us (10082us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845034.340018:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 390 00000004:00010000:2.0:1788845034.343477:0:5840:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 357427 -> 582966 00000100:00100000:2.0:1788845034.343507:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660284544:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845034.343530:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845034.343549:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad503 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ca1 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.343568:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.343571:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845034.343576:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad503 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ca1 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.343590:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773e00, rc 0) 00000100:00100000:2.0:1788845034.343647:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641600128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4662us (4852us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.343656:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1101 00000100:00100000:1.0:1788845034.346633:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.346731:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641600256 00000100:00100000:3.0:1788845034.346741:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641600256 00000100:00100000:3.0:1788845034.346801:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1102 00000100:00100000:3.0:1788845034.346811:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641600256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.346843:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.346864:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad50a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ca8 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.346939:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.346979:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383d180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660284672:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.347053:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.349772:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383d180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660284672:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845034.349798:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:3.0:1788845034.349819:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad50a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ca8 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.349850:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.349852:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845034.349858:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad50a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ca8 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.349872:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:0.0:1788845034.350170:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641600256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3357us (3545us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.350180:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1102 00000100:00100000:1.0:1788845034.353342:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.353714:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641600512 00000100:00100000:2.0:1788845034.353725:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641600512 00000100:00100000:2.0:1788845034.353845:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1103 00000100:00100000:2.0:1788845034.353856:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641600512:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845034.353950:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.353978:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad4d9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c77 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.353998:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.354002:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845034.354214:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.354229:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.354237:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.354286:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.354304:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 15 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.354318:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660284800:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845034.354675:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.359153:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.359213:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641600640 00000100:00100000:2.0:1788845034.359222:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641600640 00010000:00010000:2.0:1788845034.359400:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 16 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.359441:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 391 00000100:00100000:2.0:1788845034.359452:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641600640:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845034.359465:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.359472:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 16 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.359489:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 16 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.359511:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.359516:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.359526:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.359533:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.359550:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845034.359592:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641600640:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 143us (449us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.359598:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 391 00010000:00010000:3.0:1788845034.362265:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287ad4cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c70 expref: 15 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.362307:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660284800:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845034.362331:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.362544:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.362553:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845034.362667:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967709 00000004:00010000:2.0:1788845034.362790:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 357427 00000004:00010000:2.0:1788845034.362794:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:2.0:1788845034.362796:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845034 -> 1788845033 00000004:00010000:2.0:1788845034.362799:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:2.0:1788845034.362801:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 454 -> 454 00000004:00010000:2.0:1788845034.362809:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000661a8010 reply state 00000000427b0588 transno 4294967710 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.362818:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845034.364626:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641600512:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 10758us (11281us total) trans 4294967710 rc 0/0 00000100:00100000:2.0:1788845034.364640:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1103 00000020:00080000:2.0:1788845034.364648:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788845034.367782:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.368187:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641600768 00000100:00100000:2.0:1788845034.368197:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641600768 00000100:00100000:2.0:1788845034.368257:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 749 00000100:00100000:2.0:1788845034.368266:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641600768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.368286:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.368307:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8caf expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.368338:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.368358:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:2.0:1788845034.368363:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 11) 00010000:00010000:2.0:1788845034.368368:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.368401:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.368420:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 5 pid: 7983 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.368434:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660284928:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845034.368524:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4480 going to sleep for 11 seconds 00010000:00010000:3.0:1788845034.370075:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.370090:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.370098:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.370104:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008cba400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845034.370108:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.370116:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.370133:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.370136:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845034.370140:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.370146:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad511 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.370814:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660284928:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845034.370875:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.370920:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba800, rc 0) 00000100:00100000:3.0:1788845034.370986:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641600768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2720us (3216us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845034.370995:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 749 00000100:00100000:0.0:1788845034.375118:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.375200:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641600896 00000100:00100000:2.0:1788845034.375210:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641600896 00010000:00010000:2.0:1788845034.375252:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.375412:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 392 00000100:00100000:2.0:1788845034.375422:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641600896:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845034.375439:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.375463:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.375482:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.375510:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad3e4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1417216->1445887) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8bac expref: 6 pid: 7983 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.375524:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 11) 00010000:00010000:2.0:1788845034.375528:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 11) 00010000:00010000:2.0:1788845034.375535:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.375545:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.375582:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.375598:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660285056:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845034.375734:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.380224:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.380309:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641601024 00000100:00100000:3.0:1788845034.380319:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641601024 00002000:00010000:3.0:1788845034.380371:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1189500->18446744073709551615) 00002000:00100000:3.0:1788845034.380377:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5dca80 with x1875739641601024 ext(1189500->18446744073709551615) 00002000:00010000:3.0:1788845034.380389:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5dca80. 00000100:00100000:3.0:1788845034.380413:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 396 00000100:00100000:3.0:1788845034.380423:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641601024:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845034.380832:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1541764 -> 1189500 00002000:00010000:3.0:1788845034.380836:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00002000:00010000:3.0:1788845034.380838:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845034 -> 1788845033 00002000:00010000:3.0:1788845034.380840:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00002000:00010000:3.0:1788845034.380842:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 153 -> 81 00000100:00100000:3.0:1788845034.380920:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641601024:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 498us (709us total) trans 4294967433 rc 0/0 00000100:00100000:3.0:1788845034.380928:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 396 00002000:00010000:3.0:1788845034.380931:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1189500->18446744073709551615) 00002000:00100000:3.0:1788845034.380935:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5dca80 with x1875739641601024 ext(1189500->18446744073709551615) 00002000:00010000:3.0:1788845034.380939:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5dca80. 00000100:00100000:3.0:1788845034.383758:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.384012:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641601152 00000100:00100000:1.0:1788845034.384024:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641601152 00000100:00100000:1.0:1788845034.384080:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1104 00000100:00100000:1.0:1788845034.384090:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641601152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.384111:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.384130:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad51f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8cb6 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.384222:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad526 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.384238:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad526 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.384249:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad526 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845034.384278:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1189500 00000004:00200000:1.0:1788845034.384282:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845034.384301:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad526 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.384328:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad51f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cb6 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.384381:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad526 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cb6 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.384389:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.384392:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845034.384395:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845034.384450:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641601152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 361us (701us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.384457:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1104 00000100:00100000:2.0:1788845034.386910:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660285056:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845034.386938:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845034.387000:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641600896:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11578us (11891us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.387011:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 392 00000100:00100000:3.0:1788845034.387391:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.387791:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641601280 00000100:00100000:0.0:1788845034.387798:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641601280 00000100:00100000:0.0:1788845034.387860:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1105 00000100:00100000:0.0:1788845034.387869:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641601280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.387910:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.387931:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad52d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8cbd expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.387965:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.388004:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660285184:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.388074:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.391246:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.391347:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641601408 00000100:00100000:2.0:1788845034.391354:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641601408 00000100:00100000:2.0:1788845034.391401:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 750 00000100:00100000:2.0:1788845034.391409:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641601408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.391424:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.391440:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad534 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8cc4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.391490:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660285312:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.391560:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000004:00010000:0.0:1788845034.392147:0:5840:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 357427 -> 582966 00000004:00010000:0.0:1788845034.392150:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845033 -> 1788845034 00000100:00100000:0.0:1788845034.392181:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660285184:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.392198:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845034.392213:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad52d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cbd expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.392227:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.392230:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.392232:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad52d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cbd expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.392243:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773e00, rc 0) 00000100:00100000:0.0:1788845034.392283:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641601280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4416us (4894us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.392290:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1105 00002000:00010000:2.0:1788845034.395962:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845033 -> 1788845034 00000100:00100000:2.0:1788845034.395994:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660285312:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.396034:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad534 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cc4 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.396059:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad534 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cc4 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.396075:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba400, rc 0) 00000100:00100000:2.0:1788845034.396139:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641601408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4729us (4899us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.396147:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 750 00000100:00100000:1.0:1788845034.398962:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.399087:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641601536 00000100:00100000:0.0:1788845034.399097:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641601536 00000100:00100000:0.0:1788845034.399155:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1106 00000100:00100000:0.0:1788845034.399165:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641601536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.399184:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.399204:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad53b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ccb expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.399243:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.399591:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660285440:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.399677:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.402314:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.402502:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660285440:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.402527:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845034.402547:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad53b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ccb expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.402568:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.402572:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.402578:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287ad53b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ccb expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.402593:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773400, rc 0) 00000100:00100000:0.0:1788845034.402658:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641601536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3494us (3705us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.402667:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1106 00000100:00100000:2.0:1788845034.402857:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641601664 00000100:00100000:2.0:1788845034.402864:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641601664 00000100:00100000:2.0:1788845034.402935:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 751 00000100:00100000:2.0:1788845034.402944:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641601664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.402960:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.402977:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ad542 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8cd2 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.403030:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660285568:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.403082:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.406455:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660285568:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.406496:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ad542 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cd2 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.406521:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ad542 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cd2 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.406535:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb400, rc 0) 00000100:00100000:2.0:1788845034.406595:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641601664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3652us (4290us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.406604:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 751 00000100:00100000:3.0:1788845034.410010:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.410050:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.410160:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641601920 00000100:00100000:2.0:1788845034.410169:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641601920 00000100:00100000:2.0:1788845034.410228:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 752 00000100:00100000:2.0:1788845034.410239:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641601920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.410258:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.410276:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad549 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ce0 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.410337:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660285696:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.410425:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.410708:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641601792 00000100:00100000:0.0:1788845034.410713:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641601792 00000100:00100000:0.0:1788845034.410753:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1107 00000100:00100000:0.0:1788845034.410761:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641601792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.410775:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.410787:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad550 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8cd9 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.410816:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.410851:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660285824:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.410918:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.413601:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660285696:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.413639:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad549 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.413662:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad549 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.413677:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba400, rc 0) 00000100:00100000:2.0:1788845034.413742:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641601920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3505us (3740us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.413752:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 752 00000100:00100000:0.0:1788845034.414110:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660285824:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.414124:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845034.414136:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad550 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cd9 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.414148:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.414151:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.414156:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287ad550 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cd9 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.414167:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773e00, rc 0) 00000100:00100000:0.0:1788845034.414203:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641601792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3442us (4152us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.414210:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1107 00000100:00100000:3.0:1788845034.418871:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.419035:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641602048 00000100:00100000:1.0:1788845034.419045:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641602048 00000100:00100000:1.0:1788845034.419099:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1108 00000100:00100000:1.0:1788845034.419109:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641602048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.419131:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.419153:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ce7 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.419183:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.419192:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.419225:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.419274:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 15 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.419289:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5ddf80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660285952:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845034.419386:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.422682:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.422809:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5ddf80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660285952:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845034.422855:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.422876:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.422879:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.422912:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.422923:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:0.0:1788845034.422977:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641602048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3869us (4118us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.422986:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1108 00000100:00100000:3.0:1788845034.423087:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641602176 00000100:00100000:3.0:1788845034.423094:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641602176 00002000:00100000:3.0:1788845034.423141:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dc000 x1875739641602176/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845034.423151:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dc000 with x1875739641602176 00010000:00010000:3.0:1788845034.423158:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 16 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.423176:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 16 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845034.423185:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a03d5dc000 00000100:00100000:3.0:1788845034.423217:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 25 00000100:00100000:3.0:1788845034.423226:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641602176:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845034.426479:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:0.0:1788845034.426488:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.426490:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845034 -> 1788845033 00000004:00010000:0.0:1788845034.426493:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.426496:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 454 -> 454 00000100:00100000:0.0:1788845034.428066:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641602176:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4832us (5384us total) trans 4294967711 rc 0/0 00000100:00100000:0.0:1788845034.428079:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 25 00002000:00100000:0.0:1788845034.428093:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dc000 x1875739641602176/t4294967711(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/448 e 0 to 0 dl 1788845045 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.428103:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dc000 with x1875739641602176 00010000:00010000:0.0:1788845034.428112:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 18 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.428134:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 18 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845034.428142:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a03d5dc000 00000100:00100000:2.0:1788845034.430577:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.430682:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641602304 00000100:00100000:1.0:1788845034.430693:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641602304 00010000:00010000:1.0:1788845034.430791:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 18 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.430871:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 393 00000100:00100000:1.0:1788845034.430905:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641602304:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845034.430921:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845034.430949:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 18 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.430967:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 18 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.430989:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287ad4fc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8c9a expref: 18 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.431005:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.431011:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.431036:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.431046:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.431082:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.431095:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660286080:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845034.431177:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.434502:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.434577:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641602432 00000100:00100000:0.0:1788845034.434587:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641602432 00002000:00100000:0.0:1788845034.434638:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ed80 x1875739641602432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.434649:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ed80 with x1875739641602432 00000100:00100000:0.0:1788845034.434688:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 727 00000100:00100000:0.0:1788845034.434803:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641602432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.435053:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641602432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 355us (561us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845034.435075:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 727 00002000:00100000:0.0:1788845034.435084:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ed80 x1875739641602432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.435092:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ed80 with x1875739641602432 00000100:00100000:1.0:1788845034.437687:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660286080:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845034.437724:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845034.437792:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641602304:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6912us (7222us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.437802:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 393 00000100:00100000:2.0:1788845034.441018:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.441095:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641602560 00000100:00100000:0.0:1788845034.441104:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641602560 00002000:00100000:0.0:1788845034.441165:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251d500 x1875739641602560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.441177:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251d500 with x1875739641602560 00000100:00100000:0.0:1788845034.441214:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 728 00000100:00100000:0.0:1788845034.441223:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641602560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.441967:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641602560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 740us (955us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845034.441976:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 728 00002000:00100000:0.0:1788845034.441985:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251d500 x1875739641602560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.441994:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251d500 with x1875739641602560 00000100:00100000:2.0:1788845034.445613:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.445686:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641602688 00000100:00100000:1.0:1788845034.445706:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641602688 00002000:00100000:1.0:1788845034.445756:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251fb80 x1875739641602688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.445766:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251fb80 with x1875739641602688 00000100:00100000:1.0:1788845034.445803:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 729 00000100:00100000:1.0:1788845034.445813:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641602688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.446062:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641602688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (459us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845034.446084:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 729 00002000:00100000:1.0:1788845034.446092:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251fb80 x1875739641602688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.446099:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251fb80 with x1875739641602688 00000100:00100000:1.0:1788845034.449816:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.450220:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641602816 00000100:00100000:0.0:1788845034.450229:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641602816 00002000:00100000:0.0:1788845034.450276:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dc380 x1875739641602816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.450287:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dc380 with x1875739641602816 00000100:00100000:0.0:1788845034.450408:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 730 00000100:00100000:0.0:1788845034.450418:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641602816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.450687:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641602816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 269us (880us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845034.450702:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 730 00002000:00100000:0.0:1788845034.450709:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dc380 x1875739641602816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.450717:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dc380 with x1875739641602816 00000100:00100000:2.0:1788845034.453536:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.453619:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641602944 00000100:00100000:1.0:1788845034.453628:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641602944 00002000:00100000:1.0:1788845034.453668:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f776680 x1875739641602944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.453678:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f776680 with x1875739641602944 00000100:00100000:1.0:1788845034.453786:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 731 00000100:00100000:1.0:1788845034.453798:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641602944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.454034:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641602944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 237us (504us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845034.454041:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 731 00002000:00100000:1.0:1788845034.454049:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f776680 x1875739641602944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.454057:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f776680 with x1875739641602944 00000100:00100000:3.0:1788845034.456849:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.457443:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641603072 00000100:00100000:0.0:1788845034.457453:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641603072 00002000:00100000:0.0:1788845034.457506:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383d880 x1875739641603072/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.457517:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383d880 with x1875739641603072 00000100:00100000:0.0:1788845034.457555:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 732 00000100:00100000:0.0:1788845034.457565:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641603072:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.457792:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641603072:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 226us (976us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845034.457800:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 732 00002000:00100000:0.0:1788845034.457807:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383d880 x1875739641603072/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.457815:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383d880 with x1875739641603072 00000100:00100000:1.0:1788845034.460942:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.461162:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641603200 00000100:00100000:0.0:1788845034.461171:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641603200 00002000:00100000:0.0:1788845034.461214:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dc700 x1875739641603200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.461225:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dc700 with x1875739641603200 00000100:00100000:0.0:1788845034.461275:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 733 00000100:00100000:0.0:1788845034.461285:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641603200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.461500:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641603200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 216us (566us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845034.461509:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 733 00002000:00100000:0.0:1788845034.461515:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dc700 x1875739641603200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.461524:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dc700 with x1875739641603200 00000100:00100000:1.0:1788845034.464835:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.464958:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641603328 00000100:00100000:3.0:1788845034.464967:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641603328 00002000:00100000:3.0:1788845034.465013:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5ddf80 x1875739641603328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845034.465025:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5ddf80 with x1875739641603328 00000100:00100000:3.0:1788845034.465069:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 734 00000100:00100000:3.0:1788845034.465077:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641603328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845034.465269:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641603328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 192us (450us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845034.465277:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 734 00002000:00100000:3.0:1788845034.465283:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5ddf80 x1875739641603328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845034.465290:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5ddf80 with x1875739641603328 00000100:00100000:3.0:1788845034.468514:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.468925:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641603456 00000100:00100000:1.0:1788845034.468935:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641603456 00002000:00100000:1.0:1788845034.468981:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383c380 x1875739641603456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.469014:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c380 with x1875739641603456 00000100:00100000:1.0:1788845034.469053:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 735 00000100:00100000:1.0:1788845034.469063:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641603456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.469278:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641603456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 216us (773us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845034.469287:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 735 00002000:00100000:1.0:1788845034.469294:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383c380 x1875739641603456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.469302:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c380 with x1875739641603456 00000100:00100000:1.0:1788845034.473092:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.473508:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641603584 00000100:00100000:3.0:1788845034.473518:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641603584 00002000:00100000:3.0:1788845034.473565:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dce00 x1875739641603584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845034.473577:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dce00 with x1875739641603584 00000100:00100000:3.0:1788845034.473615:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 736 00000100:00100000:3.0:1788845034.473625:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641603584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845034.474126:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641603584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 499us (1042us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845034.474134:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 736 00002000:00100000:3.0:1788845034.474142:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dce00 x1875739641603584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845034.474150:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dce00 with x1875739641603584 00000100:00100000:3.0:1788845034.477662:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.477788:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641603712 00000100:00100000:1.0:1788845034.477800:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641603712 00002000:00100000:1.0:1788845034.477855:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ed80 x1875739641603712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.477867:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ed80 with x1875739641603712 00000100:00100000:2.0:1788845034.477973:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 737 00000100:00100000:2.0:1788845034.477985:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641603712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.478192:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641603712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 210us (539us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845034.478200:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 737 00002000:00100000:2.0:1788845034.478206:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358ed80 x1875739641603712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.478214:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358ed80 with x1875739641603712 00000100:00100000:1.0:1788845034.481457:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.481570:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641603840 00000100:00100000:0.0:1788845034.481579:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641603840 00002000:00100000:0.0:1788845034.481628:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918e00 x1875739641603840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.481638:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918e00 with x1875739641603840 00000100:00100000:0.0:1788845034.481681:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 738 00000100:00100000:0.0:1788845034.481692:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2367:x1875739641603840:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.481971:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2367:x1875739641603840:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (523us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845034.481979:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 738 00002000:00100000:0.0:1788845034.481987:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918e00 x1875739641603840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.481994:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918e00 with x1875739641603840 00000100:00100000:3.0:1788845034.485669:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.485799:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641603968 00000100:00100000:2.0:1788845034.485809:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641603968 00002000:00100000:2.0:1788845034.485871:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d880 x1875739641603968/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.485920:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d880 with x1875739641603968 00000100:00100000:2.0:1788845034.485958:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 739 00000100:00100000:2.0:1788845034.485968:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641603968:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.486192:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641603968:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 224us (531us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845034.486200:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 739 00002000:00100000:2.0:1788845034.486207:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358d880 x1875739641603968/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.486215:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358d880 with x1875739641603968 00000100:00100000:3.0:1788845034.490174:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.490419:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641604096 00000100:00100000:0.0:1788845034.490429:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641604096 00002000:00100000:0.0:1788845034.490478:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1f80 x1875739641604096/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.490502:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1f80 with x1875739641604096 00000100:00100000:0.0:1788845034.490541:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 740 00000100:00100000:0.0:1788845034.490551:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641604096:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.490774:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641604096:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 223us (610us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845034.490783:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 740 00002000:00100000:0.0:1788845034.490790:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1f80 x1875739641604096/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.490798:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1f80 with x1875739641604096 00000100:00100000:3.0:1788845034.494987:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.495356:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641604224 00000100:00100000:2.0:1788845034.495366:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641604224 00002000:00100000:2.0:1788845034.495413:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb2680 x1875739641604224/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.495425:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb2680 with x1875739641604224 00000100:00100000:2.0:1788845034.495514:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 741 00000100:00100000:2.0:1788845034.495524:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641604224:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.495792:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641604224:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 269us (816us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845034.495800:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 741 00002000:00100000:2.0:1788845034.495807:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb2680 x1875739641604224/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.495831:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb2680 with x1875739641604224 00000100:00100000:3.0:1788845034.499129:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.499309:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641604352 00000100:00100000:0.0:1788845034.499319:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641604352 00002000:00100000:0.0:1788845034.499379:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7c700 x1875739641604352/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.499390:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7c700 with x1875739641604352 00000100:00100000:0.0:1788845034.499429:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 742 00000100:00100000:0.0:1788845034.499440:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641604352:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845034.502634:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:0.0:1788845034.502643:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.502646:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:0.0:1788845034.502648:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.502651:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 454 -> 454 00000100:00100000:0.0:1788845034.502732:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641604352:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3289us (3608us total) trans 4294967712 rc 0/0 00000100:00100000:0.0:1788845034.502755:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 742 00002000:00100000:0.0:1788845034.502767:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7c700 x1875739641604352/t4294967712(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/448 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.502776:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7c700 with x1875739641604352 00000100:00100000:3.0:1788845034.505400:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.505496:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641604480 00000100:00100000:2.0:1788845034.505505:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641604480 00000100:00100000:2.0:1788845034.505558:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1109 00000100:00100000:2.0:1788845034.505567:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641604480:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:2.0:1788845034.530484:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641604480:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 24767us (24940us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.530502:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1109 00000001:00080000:2.0:1788845034.530543:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967712 is committed 00000001:00080000:2.0:1788845034.530554:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967711 is committed 00000100:00100000:3.0:1788845034.533546:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.533714:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641604608 00000100:00100000:0.0:1788845034.533724:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641604608 00000100:00100000:0.0:1788845034.533784:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 753 00000100:00100000:0.0:1788845034.533794:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641604608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.533813:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.533841:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000034594ef8/0xb2187301287ad55e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8cee expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.533946:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660286208:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.534033:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.536582:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660286208:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845034.536624:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000034594ef8/0xb2187301287ad55e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cee expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.536649:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034594ef8/0xb2187301287ad55e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cee expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.536661:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773400, rc 0) 00000100:00100000:0.0:1788845034.536731:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641604608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2937us (3194us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.536758:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 753 00000100:00100000:3.0:1788845034.539476:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.539622:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641604736 00000100:00100000:2.0:1788845034.539633:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641604736 00000100:00100000:2.0:1788845034.539689:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 754 00000100:00100000:2.0:1788845034.539708:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641604736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.539728:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.539744:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad565 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8cf5 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.539800:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660286336:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.539927:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.543132:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660286336:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.543176:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad565 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cf5 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.543208:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad565 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cf5 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.543222:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbac00, rc 0) 00000100:00100000:2.0:1788845034.543285:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641604736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3586us (3818us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.543293:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 754 00000100:00100000:0.0:1788845034.546490:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.546646:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641604864 00000100:00100000:2.0:1788845034.546657:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641604864 00000100:00100000:2.0:1788845034.546722:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 755 00000100:00100000:2.0:1788845034.546732:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641604864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.546753:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.546774:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ad56c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8cfc expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.546845:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660286464:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.546951:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.549632:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660286464:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.549672:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ad56c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cfc expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.549706:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f9c16dda/0xb2187301287ad56c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cfc expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.549730:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb000, rc 0) 00000100:00100000:2.0:1788845034.549791:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641604864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3060us (3311us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.549800:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 755 00000100:00100000:3.0:1788845034.552876:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.553052:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641604992 00000100:00100000:2.0:1788845034.553062:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641604992 00000100:00100000:2.0:1788845034.553118:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 756 00000100:00100000:2.0:1788845034.553127:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641604992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.553147:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.553165:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad573 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d03 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.553229:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660286592:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.553318:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.555856:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660286592:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.555915:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad573 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d03 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.555939:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad573 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d03 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.555954:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbac00, rc 0) 00000100:00100000:2.0:1788845034.556009:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641604992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2882us (3143us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.556018:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 756 00000100:00100000:1.0:1788845034.558957:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.559236:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641605120 00000100:00100000:2.0:1788845034.559246:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641605120 00000100:00100000:2.0:1788845034.559373:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 743 00000100:00100000:2.0:1788845034.559383:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641605120:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845034.559664:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:2.0:1788845034.559669:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:2.0:1788845034.559672:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845033 -> 1788845033 00000004:00010000:2.0:1788845034.559674:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:2.0:1788845034.559677:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 454 -> 518 00000100:00100000:2.0:1788845034.559987:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641605120:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 604us (1038us total) trans 4294967713 rc 0/0 00000100:00100000:2.0:1788845034.559996:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 743 00000100:00100000:3.0:1788845034.562374:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.562653:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641605248 00000100:00100000:1.0:1788845034.562664:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641605248 00000100:00100000:1.0:1788845034.562727:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1110 00000100:00100000:1.0:1788845034.562737:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641605248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.562762:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.562783:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad57a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d0a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.562909:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.562926:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.562939:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845034.562966:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1189500 00000004:00200000:1.0:1788845034.562970:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845034.562988:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.563015:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad57a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.563034:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.563043:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.563046:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845034.563049:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845034.563101:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641605248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 366us (737us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.563109:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1110 00000100:00100000:3.0:1788845034.565642:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.565711:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641605376 00000100:00100000:0.0:1788845034.565720:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641605376 00000100:00100000:0.0:1788845034.565760:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1111 00000100:00100000:0.0:1788845034.565766:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641605376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.565786:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.565806:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad588 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d11 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.565849:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.565917:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660286720:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.565988:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000004:00010000:0.0:1788845034.569037:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845033 -> 1788845034 00000100:00100000:0.0:1788845034.569070:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660286720:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.569094:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845034.569114:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad588 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d11 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.569135:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.569138:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.569143:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287ad588 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d11 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.569179:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772a00, rc 0) 00000100:00100000:0.0:1788845034.569243:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641605376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3475us (3607us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.569252:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1111 00000100:00100000:0.0:1788845034.572796:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.572925:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641605504 00000100:00100000:1.0:1788845034.572936:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641605504 00000100:00100000:1.0:1788845034.573007:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1112 00000100:00100000:1.0:1788845034.573017:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641605504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.573040:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.573059:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad58f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d18 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.573095:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.573135:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660286848:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.573226:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.575990:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660286848:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.576018:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845034.576037:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad58f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d18 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.576057:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.576060:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.576066:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad58f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d18 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.576082:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845034.576140:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641605504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3124us (3353us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.576149:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1112 00000100:00100000:2.0:1788845034.579865:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.580004:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641605632 00000100:00100000:0.0:1788845034.580015:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641605632 00000100:00100000:0.0:1788845034.580074:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 757 00000100:00100000:0.0:1788845034.580085:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641605632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.580105:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.580123:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e62a8a6f/0xb2187301287ad596 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d1f expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.580185:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660286976:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.580276:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.582589:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660286976:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845034.582630:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e62a8a6f/0xb2187301287ad596 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d1f expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.582657:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e62a8a6f/0xb2187301287ad596 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d1f expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.582671:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772a00, rc 0) 00000100:00100000:3.0:1788845034.582742:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641605632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2659us (2888us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.582751:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 757 00000100:00100000:3.0:1788845034.587029:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.587179:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641605760 00000100:00100000:2.0:1788845034.587188:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641605760 00002000:00100000:2.0:1788845034.587237:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7d180 x1875739641605760/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 520/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.587249:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7d180 with x1875739641605760 00000100:00100000:2.0:1788845034.587288:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 744 00000100:00100000:2.0:1788845034.587297:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641605760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.595708:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641605760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8397us (8674us total) trans 0 rc 524288/524288 00000100:00100000:2.0:1788845034.595721:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 744 00002000:00100000:2.0:1788845034.595735:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7d180 x1875739641605760/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 520/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 524288/524288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.595746:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7d180 with x1875739641605760 00000100:00100000:2.0:1788845034.598637:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.598948:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641605888 00000100:00100000:0.0:1788845034.598961:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641605888 00002000:00100000:0.0:1788845034.599012:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f775880 x1875739641605888/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.599023:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f775880 with x1875739641605888 00000100:00100000:0.0:1788845034.599061:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 745 00000100:00100000:0.0:1788845034.599071:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641605888:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.599325:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641605888:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (697us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.599349:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 745 00002000:00100000:0.0:1788845034.599357:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f775880 x1875739641605888/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.599365:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f775880 with x1875739641605888 00000100:00100000:3.0:1788845034.601828:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.601970:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641606016 00000100:00100000:0.0:1788845034.601980:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641606016 00002000:00100000:0.0:1788845034.602024:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7f480 x1875739641606016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.602035:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7f480 with x1875739641606016 00000100:00100000:0.0:1788845034.602072:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 746 00000100:00100000:0.0:1788845034.602082:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641606016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.602316:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641606016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 235us (504us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.602323:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 746 00002000:00100000:0.0:1788845034.602330:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7f480 x1875739641606016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.602348:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7f480 with x1875739641606016 00000100:00100000:3.0:1788845034.604562:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.604689:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641606144 00000100:00100000:0.0:1788845034.604707:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641606144 00002000:00100000:0.0:1788845034.604749:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7d880 x1875739641606144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.604760:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7d880 with x1875739641606144 00000100:00100000:0.0:1788845034.604799:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 747 00000100:00100000:0.0:1788845034.604809:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.605101:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (546us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.605109:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 747 00002000:00100000:0.0:1788845034.605116:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7d880 x1875739641606144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.605124:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7d880 with x1875739641606144 00000100:00100000:1.0:1788845034.607680:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.608109:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641606272 00000100:00100000:0.0:1788845034.608119:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641606272 00002000:00100000:0.0:1788845034.608174:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918a80 x1875739641606272/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.608187:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918a80 with x1875739641606272 00000100:00100000:0.0:1788845034.608230:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 748 00000100:00100000:0.0:1788845034.608241:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641606272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.608702:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641606272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 453us (1022us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.608711:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 748 00002000:00100000:0.0:1788845034.608719:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918a80 x1875739641606272/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.608727:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918a80 with x1875739641606272 00000100:00100000:3.0:1788845034.611787:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.611938:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641606400 00000100:00100000:0.0:1788845034.611947:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641606400 00002000:00100000:0.0:1788845034.611996:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7f100 x1875739641606400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.612007:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7f100 with x1875739641606400 00000100:00100000:0.0:1788845034.612062:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 749 00000100:00100000:0.0:1788845034.612072:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.612312:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 240us (536us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.612319:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 749 00002000:00100000:0.0:1788845034.612325:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7f100 x1875739641606400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.612699:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7f100 with x1875739641606400 00000100:00100000:2.0:1788845034.614708:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.615074:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641606528 00000100:00100000:0.0:1788845034.615085:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641606528 00002000:00100000:0.0:1788845034.615133:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f775c00 x1875739641606528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.615144:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f775c00 with x1875739641606528 00000100:00100000:0.0:1788845034.615185:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 750 00000100:00100000:0.0:1788845034.615195:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641606528:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.615578:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641606528:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 382us (885us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.615586:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 750 00002000:00100000:0.0:1788845034.615594:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f775c00 x1875739641606528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.615601:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f775c00 with x1875739641606528 00000100:00100000:1.0:1788845034.621116:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.621231:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641606656 00000100:00100000:0.0:1788845034.621242:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641606656 00002000:00100000:0.0:1788845034.621287:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919f80 x1875739641606656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.621310:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919f80 with x1875739641606656 00000100:00100000:2.0:1788845034.621437:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 751 00000100:00100000:2.0:1788845034.621449:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606656:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.622259:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606656:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 809us (1152us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.622268:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 751 00002000:00100000:2.0:1788845034.622277:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919f80 x1875739641606656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.622286:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919f80 with x1875739641606656 00000100:00100000:3.0:1788845034.624871:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.625197:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641606784 00000100:00100000:0.0:1788845034.625206:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641606784 00002000:00100000:0.0:1788845034.625254:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7ed80 x1875739641606784/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.625266:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7ed80 with x1875739641606784 00000100:00100000:0.0:1788845034.625350:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 752 00000100:00100000:0.0:1788845034.625360:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606784:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.625599:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606784:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 239us (736us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.625607:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 752 00002000:00100000:0.0:1788845034.625614:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7ed80 x1875739641606784/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.625622:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7ed80 with x1875739641606784 00000100:00100000:3.0:1788845034.627937:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.628036:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641606912 00000100:00100000:2.0:1788845034.628047:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641606912 00002000:00100000:2.0:1788845034.628088:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7ce00 x1875739641606912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.628099:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7ce00 with x1875739641606912 00000100:00100000:2.0:1788845034.628122:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 753 00000100:00100000:2.0:1788845034.628131:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.628355:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641606912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 224us (425us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.628363:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 753 00002000:00100000:2.0:1788845034.628371:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7ce00 x1875739641606912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.628384:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7ce00 with x1875739641606912 00000100:00100000:2.0:1788845034.631447:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.631824:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641607040 00000100:00100000:0.0:1788845034.631834:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641607040 00000100:00100000:0.0:1788845034.631911:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1113 00000100:00100000:0.0:1788845034.631921:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641607040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.631941:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.631960:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287ad59d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d26 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.631996:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.632053:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660287104:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.632132:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.634272:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660287104:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.634299:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845034.634317:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287ad59d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d26 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.634399:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.634403:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.634409:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287ad59d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d26 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.634425:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772c00, rc 0) 00000100:00100000:0.0:1788845034.634482:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641607040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2563us (3044us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.634490:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1113 00000100:00100000:2.0:1788845034.637325:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.637731:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641607168 00000100:00100000:0.0:1788845034.637740:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641607168 00000100:00100000:0.0:1788845034.637802:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1114 00000100:00100000:0.0:1788845034.637812:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641607168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.637842:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.637863:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287ad5a4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d2d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.637917:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.637972:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660287232:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845034.638524:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.640980:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660287232:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.641009:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845034.641026:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287ad5a4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d2d expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.641046:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.641050:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.641054:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287ad5a4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d2d expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.641068:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773000, rc 0) 00000100:00100000:1.0:1788845034.641308:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641607168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3495us (3991us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.641316:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1114 00000100:00100000:0.0:1788845034.645526:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.645626:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641607296 00000100:00100000:2.0:1788845034.645636:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641607296 00000100:00100000:2.0:1788845034.645729:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1115 00000100:00100000:2.0:1788845034.645738:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641607296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.645760:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.645781:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d34 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.645933:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.645945:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.645980:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.645999:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 11 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.646014:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f775180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660287360:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845034.646087:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.649242:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f775180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660287360:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845034.649277:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.649297:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.649300:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.649305:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.649316:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbaa00, rc 0) 00000100:00100000:0.0:1788845034.649381:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641607296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3643us (3864us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.649390:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1115 00000100:00100000:3.0:1788845034.655181:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.655522:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641607424 00000100:00100000:2.0:1788845034.655533:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641607424 00010000:00010000:2.0:1788845034.655570:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 12 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.655667:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 394 00000100:00100000:2.0:1788845034.655677:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641607424:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845034.655701:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.655728:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 12 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.655746:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 12 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.655769:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad557 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ce7 expref: 12 pid: 5841 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.655787:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.655802:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.655824:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.655832:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.655870:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.656095:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660287488:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845034.656181:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.658236:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660287488:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845034.658270:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845034.658325:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641607424:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2648us (3153us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845034.658343:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 394 00000100:00100000:3.0:1788845034.666098:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.666232:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641607552 00000100:00100000:0.0:1788845034.666243:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641607552 00002000:00100000:0.0:1788845034.666309:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7e680 x1875739641607552/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.666320:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7e680 with x1875739641607552 00000100:00100000:0.0:1788845034.666366:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 754 00000100:00100000:0.0:1788845034.666376:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641607552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845034.670557:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.674640:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641607680 00000100:00100000:3.0:1788845034.674656:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641607680 00002000:00100000:3.0:1788845034.674722:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1191935) req@ffff94a035b7c380 x1875739641607680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845034.674735:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7c380 with x1875739641607680 ext(1048576->1191935) 00002000:00010000:3.0:1788845034.674741:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7c380 00000100:00100000:3.0:1788845034.674835:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 397 00000100:00100000:3.0:1788845034.674845:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641607680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.684374:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641607552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17989us (18276us total) trans 0 rc 585728/585728 00000100:00100000:1.0:1788845034.684385:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 754 00002000:00100000:1.0:1788845034.684397:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7e680 x1875739641607552/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 585728/585728 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.684405:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7e680 with x1875739641607552 00000100:00100000:2.0:1788845034.688812:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641607680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 13957us (18251us total) trans 0 rc 143360/143360 00000100:00100000:2.0:1788845034.688834:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 397 00002000:00100000:2.0:1788845034.688847:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1191935) req@ffff94a035b7c380 x1875739641607680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 143360/143360 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.688857:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7c380 with x1875739641607680 ext(1048576->1191935) 00002000:00010000:2.0:1788845034.688863:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7c380 00000100:00100000:1.0:1788845034.690509:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.690612:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.690639:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641607936 00000100:00100000:1.0:1788845034.690647:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641607936 00000100:00100000:1.0:1788845034.690693:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 758 00000100:00100000:1.0:1788845034.690709:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641607936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.690738:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.690755:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad5b2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d42 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.691565:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008919c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660287616:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.691649:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.692270:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641607808 00000100:00100000:2.0:1788845034.692277:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641607808 00000100:00100000:2.0:1788845034.692329:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1116 00000100:00100000:2.0:1788845034.692350:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641607808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.692367:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.692384:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ad5b9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d3b expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.692416:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ad5b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d3b expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845034.692428:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845034.692435:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ad5b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d3b expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.692448:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.692452:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845034.692455:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbac00, rc 0) 00000100:00100000:2.0:1788845034.692488:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641607808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 141us (1986us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.692495:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1116 00000100:00100000:3.0:1788845034.697093:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008919c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660287616:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845034.697135:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad5b2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d42 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.697160:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad5b2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d42 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.697174:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:3.0:1788845034.697236:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641607936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6533us (6622us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.697261:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 758 00000100:00100000:0.0:1788845034.700169:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.700292:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641608064 00000100:00100000:2.0:1788845034.700302:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641608064 00000100:00100000:2.0:1788845034.700463:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 759 00000100:00100000:2.0:1788845034.700475:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641608064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.700494:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.700512:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ad5c0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d49 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.700570:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f777100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660287744:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.700878:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.707197:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f777100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660287744:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.707240:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ad5c0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d49 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.707264:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8dcf1b0/0xb2187301287ad5c0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d49 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.707281:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbbc00, rc 0) 00000100:00100000:2.0:1788845034.707458:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641608064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6984us (7300us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.707466:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 759 00000100:00100000:2.0:1788845034.712585:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.713845:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641608320 00000100:00100000:0.0:1788845034.713859:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641608320 00000100:00100000:0.0:1788845034.714077:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1117 00000100:00100000:0.0:1788845034.714089:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641608320:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845034.714148:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845034.714176:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad526 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8cb6 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.714194:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.714199:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845034.714447:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.714462:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.714471:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.714502:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.714520:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 14 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.714535:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660287872:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845034.714624:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.717822:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.717945:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660287872:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845034.717972:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.718038:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641608448 00000100:00100000:3.0:1788845034.718046:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641608448 00010000:00010000:3.0:1788845034.718079:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 15 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.718117:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 395 00000100:00100000:3.0:1788845034.718127:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641608448:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845034.718140:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845034.718148:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 15 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.718165:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 15 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.718187:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad581 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d0a expref: 15 pid: 11036 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.718201:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.718205:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.718215:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.718222:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.718266:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845034.718314:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641608448:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 188us (510us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845034.718322:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 395 00010000:00010000:0.0:1788845034.718670:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.718678:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845034.718800:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967714 00000004:00010000:0.0:1788845034.718861:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:0.0:1788845034.718865:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.718868:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845034 -> 1788845033 00000004:00010000:0.0:1788845034.718870:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:0.0:1788845034.718872:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:0.0:1788845034.718879:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000b92d085 reply state 0000000060aa74a3 transno 4294967715 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.718911:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845034.718984:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641608320:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4897us (6406us total) trans 4294967715 rc 0/0 00000100:00100000:0.0:1788845034.718991:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1117 00000020:00080000:0.0:1788845034.718995:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845034.720959:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.720976:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.720985:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.720991:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845034.720996:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.721005:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.721021:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.721024:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845034.721029:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.721036:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287ad5c7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.722860:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.723853:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641608576 00000100:00100000:2.0:1788845034.723864:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641608576 00000100:00100000:1.0:1788845034.724231:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1118 00000100:00100000:1.0:1788845034.724242:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641608576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.724282:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.724286:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788845034.724331:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ad5b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d3b expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.724359:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.724364:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845034.724389:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d50 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.724416:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.724425:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.724476:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.724493:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 13 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.724507:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660288000:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845034.724593:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.729217:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660288000:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845034.729256:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.729275:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845034.729278:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845034.729283:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.729294:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:2.0:1788845034.729363:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641608576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5124us (6512us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.729373:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1118 00000100:00100000:3.0:1788845034.730447:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.731063:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641608704 00000100:00100000:0.0:1788845034.731074:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641608704 00010000:00010000:0.0:1788845034.731113:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 14 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.731155:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 396 00000100:00100000:0.0:1788845034.731165:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641608704:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845034.731178:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845034.731206:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 14 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.731232:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 14 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.731267:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad5ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d34 expref: 14 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.731287:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.731294:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.731305:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.731315:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.731454:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.731469:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660288128:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845034.731578:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.738232:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.738482:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641608832 00000100:00100000:1.0:1788845034.738492:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641608832 00000100:00100000:1.0:1788845034.738549:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 760 00000100:00100000:1.0:1788845034.738560:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641608832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.738580:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.738597:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d57 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.738628:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.738638:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 11) 00010000:00010000:1.0:1788845034.738643:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 11) 00010000:00010000:1.0:1788845034.738648:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.738682:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.738725:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 6 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.738740:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660288256:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845034.738839:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.738877:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660288128:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845034.738914:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845034.738940:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641608704:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7776us (8497us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.738949:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 396 00000100:00100000:2.0:1788845034.742402:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.742503:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660288256:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845034.742549:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.742570:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:1.0:1788845034.742624:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641608832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4066us (4403us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.742633:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 760 00000100:00100000:3.0:1788845034.747243:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641608960 00000100:00100000:3.0:1788845034.747311:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641608960 00010000:00010000:0.0:1788845034.747500:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 7 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.747583:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 397 00000100:00100000:0.0:1788845034.747607:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d49850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641608960:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845034.747623:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845034.747646:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 7 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.747664:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 7 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.747699:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000977cc89d/0xb2187301287ad518 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8caf expref: 7 pid: 6685 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.747717:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 11) 00010000:00010000:0.0:1788845034.747721:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 11) 00010000:00010000:0.0:1788845034.747730:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.747741:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.747790:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.747806:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660288384:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845034.747967:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.752094:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.752225:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660288384:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845034.752275:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845034.752341:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d49850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641608960:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4728us (9940us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.752353:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 397 00000100:00100000:1.0:1788845034.753506:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641609088 00000100:00100000:1.0:1788845034.753516:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641609088 00002000:00010000:1.0:1788845034.753558:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1204696->18446744073709551615) 00002000:00100000:1.0:1788845034.753562:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f777b80 with x1875739641609088 ext(1204696->18446744073709551615) 00002000:00010000:1.0:1788845034.753574:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f777b80. 00000100:00100000:1.0:1788845034.753658:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 398 00000100:00100000:1.0:1788845034.753670:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641609088:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845034.753922:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1189500 -> 1204696 00002000:00010000:1.0:1788845034.753926:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00002000:00010000:1.0:1788845034.753929:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845034 -> 1788845033 00002000:00010000:1.0:1788845034.753931:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00002000:00010000:1.0:1788845034.753934:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 81 -> 81 00000100:00100000:1.0:1788845034.753987:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641609088:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 320us (1904us total) trans 4294967434 rc 0/0 00000100:00100000:1.0:1788845034.753995:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 398 00002000:00010000:1.0:1788845034.753998:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1204696->18446744073709551615) 00002000:00100000:1.0:1788845034.754001:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f777b80 with x1875739641609088 ext(1204696->18446744073709551615) 00002000:00010000:1.0:1788845034.754006:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f777b80. 00000100:00100000:2.0:1788845034.757179:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.757275:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641609216 00000100:00100000:1.0:1788845034.757285:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641609216 00000100:00100000:1.0:1788845034.757353:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1119 00000100:00100000:1.0:1788845034.757363:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641609216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.757385:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.757406:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad5dc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d5e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.757492:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.757506:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.757537:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845034.757565:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1204696 00000004:00200000:1.0:1788845034.757569:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845034.757586:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.757610:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad5dc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.757628:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.757637:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.757639:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845034.757643:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845034.757711:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641609216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 348us (541us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.757718:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1119 00000100:00100000:2.0:1788845034.760509:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.760594:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641609344 00000100:00100000:1.0:1788845034.760603:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641609344 00000100:00100000:1.0:1788845034.760662:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1120 00000100:00100000:1.0:1788845034.760673:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641609344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.760704:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.760725:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad5ea lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d65 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.760784:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.760837:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660288512:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.760952:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.762953:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.763043:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641609472 00000100:00100000:3.0:1788845034.763053:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641609472 00000100:00100000:3.0:1788845034.763107:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 761 00000100:00100000:3.0:1788845034.763118:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641609472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.763136:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.763151:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad5f1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d6c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.763206:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660288640:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.763285:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000004:00010000:1.0:1788845034.766109:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845033 -> 1788845034 00000100:00100000:1.0:1788845034.766144:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660288512:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.766169:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845034.766188:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad5ea lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d65 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.766208:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.766211:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.766216:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad5ea lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d65 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.766231:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845034.766291:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641609344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5620us (5790us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.766300:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1120 00002000:00010000:3.0:1788845034.769527:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845033 -> 1788845034 00000100:00100000:3.0:1788845034.769558:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660288640:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845034.769595:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad5f1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d6c expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.769622:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad5f1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d6c expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.769635:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:3.0:1788845034.769702:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641609472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6577us (6749us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.769711:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 761 00000100:00100000:2.0:1788845034.773177:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.773266:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641609600 00000100:00100000:1.0:1788845034.773275:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641609600 00000100:00100000:1.0:1788845034.773345:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1121 00000100:00100000:1.0:1788845034.773355:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641609600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.773376:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.773409:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad5f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d73 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.773446:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.773486:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660288768:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.773570:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.775489:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.775567:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641609728 00000100:00100000:3.0:1788845034.775578:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641609728 00000100:00100000:3.0:1788845034.775654:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 762 00000100:00100000:3.0:1788845034.775663:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641609728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.775681:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.775708:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad5ff lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d7a expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.775766:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660288896:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.775848:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.778534:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660288896:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845034.778570:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad5ff lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d7a expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.778595:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad5ff lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d7a expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.778609:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ea00, rc 0) 00000100:00100000:3.0:1788845034.778667:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641609728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3004us (3185us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.778675:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 762 00000100:00100000:1.0:1788845034.779241:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660288768:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.779258:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845034.779270:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad5f8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d73 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.779285:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.779289:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.779293:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad5f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d73 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.779305:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845034.779330:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641609600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5977us (6165us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.779350:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1121 00000100:00100000:2.0:1788845034.784030:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.784110:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641609856 00000100:00100000:1.0:1788845034.784120:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641609856 00000100:00100000:1.0:1788845034.784183:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1122 00000100:00100000:1.0:1788845034.784194:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641609856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.784215:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.784249:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d81 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.784360:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.784372:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.784406:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.784425:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 12 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.784439:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660289024:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845034.784524:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.787066:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660289024:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845034.787102:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.787122:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.787125:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.787131:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.787141:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845034.787215:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641609856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3023us (3196us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.787225:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1122 00000100:00100000:0.0:1788845034.789425:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.789551:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641609984 00000100:00100000:3.0:1788845034.789560:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641609984 00010000:00010000:3.0:1788845034.789595:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 13 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.789634:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 398 00000100:00100000:3.0:1788845034.789644:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641609984:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845034.789673:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845034.789707:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 13 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.789725:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 13 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.789746:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad5ce lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d50 expref: 13 pid: 5839 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.789763:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.789769:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.789780:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.789791:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.789837:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.789852:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660289152:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845034.789974:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.793039:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.793132:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660289152:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845034.793171:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845034.793223:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641609984:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3580us (3805us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845034.793234:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 398 00000100:00100000:1.0:1788845034.793310:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641610112 00000100:00100000:1.0:1788845034.793316:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641610112 00000100:00100000:1.0:1788845034.793380:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 755 00000100:00100000:1.0:1788845034.793388:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641610112:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845034.793556:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:1.0:1788845034.793560:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845034 -> 1788845034 00000004:00010000:1.0:1788845034.793562:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845034 -> 1788845034 00000004:00010000:1.0:1788845034.793565:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845034 -> 1788845034 00000004:00010000:1.0:1788845034.793567:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:1.0:1788845034.793596:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641610112:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 209us (567us total) trans 4294967716 rc 0/0 00000100:00100000:1.0:1788845034.793602:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 755 00000100:00100000:2.0:1788845034.797426:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.797509:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641610240 00000100:00100000:1.0:1788845034.797518:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641610240 00000100:00100000:1.0:1788845034.797571:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 763 00000100:00100000:1.0:1788845034.797582:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641610240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.797601:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.797619:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad60d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d88 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.797675:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660289280:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.797762:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.800265:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660289280:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845034.800321:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad60d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d88 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.800357:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad60d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d88 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.800371:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845034.800428:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641610240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2848us (3010us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.800437:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 763 00000100:00100000:2.0:1788845034.803973:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.804105:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641610368 00000100:00100000:1.0:1788845034.804115:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641610368 00000100:00100000:1.0:1788845034.804171:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 764 00000100:00100000:1.0:1788845034.804181:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641610368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.804199:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.804218:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad614 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d8f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.804277:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660289408:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.804363:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.807314:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660289408:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845034.807418:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad614 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d8f expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.807443:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad614 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d8f expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.807472:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845034.807527:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641610368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3347us (3563us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.807536:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 764 00000100:00100000:0.0:1788845034.811112:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.811229:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641610496 00000100:00100000:3.0:1788845034.811241:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641610496 00000100:00100000:3.0:1788845034.811306:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 765 00000100:00100000:3.0:1788845034.811317:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641610496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.811343:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.811363:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d96 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.811395:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.811403:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(8 11) 00010000:00010000:3.0:1788845034.811407:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 8) 00010000:00010000:3.0:1788845034.811413:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1310720->1372159] (req 1310720->1372159) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.811450:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.811471:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 6 pid: 7983 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.811508:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660289536:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845034.811590:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.814567:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660289536:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845034.814602:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1310720->1372159] (req 1310720->1372159) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.814624:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ea00, rc 0) 00000100:00100000:3.0:1788845034.814734:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641610496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3418us (3634us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845034.814743:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 765 00000100:00100000:0.0:1788845034.816118:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.817067:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641610624 00000100:00100000:1.0:1788845034.817080:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641610624 00010000:00010000:1.0:1788845034.817122:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 7 pid: 7983 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.817210:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 399 00000100:00100000:1.0:1788845034.817221:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641610624:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845034.817234:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845034.817259:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 7 pid: 7983 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.817278:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 7 pid: 7983 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.817303:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad5d5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d57 expref: 7 pid: 7983 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.817318:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 8) 00010000:00010000:1.0:1788845034.817323:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 8) 00010000:00010000:1.0:1788845034.817343:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.817370:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.817411:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.817426:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660289664:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845034.817511:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.823950:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660289664:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845034.823980:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845034.824192:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641610624:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6971us (8079us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.824204:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 399 00000100:00100000:0.0:1788845034.824408:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.824532:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641610752 00000100:00100000:1.0:1788845034.824539:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641610752 00000100:00100000:1.0:1788845034.824593:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1123 00000100:00100000:1.0:1788845034.824602:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641610752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.824624:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.824644:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ad622 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8d9d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.824731:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad629 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.824747:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad629 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.824775:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad629 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845034.824803:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1204696 00000004:00200000:1.0:1788845034.824807:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845034.824837:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad629 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.824863:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ad622 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d9d expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.824913:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad629 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d9d expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.824924:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.824927:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845034.824931:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845034.824979:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641610752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 377us (571us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845034.824985:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1123 00000100:00100000:2.0:1788845034.830020:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.830291:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.830401:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641611008 00000100:00100000:3.0:1788845034.830413:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641611008 00000100:00100000:3.0:1788845034.830479:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 766 00000100:00100000:3.0:1788845034.830491:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641611008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.830510:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.830528:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad630 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8dab expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.830590:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660289792:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.830679:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.832879:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641610880 00000100:00100000:1.0:1788845034.833252:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641610880 00000100:00100000:1.0:1788845034.833315:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1124 00000100:00100000:1.0:1788845034.833325:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641610880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.833346:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.833367:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ad637 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8da4 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.833420:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.833466:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660289920:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.833656:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00002000:00010000:0.0:1788845034.834561:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1204696 -> 1368789 00000100:00100000:0.0:1788845034.834588:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660289792:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845034.834622:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad630 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dab expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.834646:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093b5acb4/0xb2187301287ad630 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dab expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.834658:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:0.0:1788845034.834726:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641611008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4237us (4434us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.834735:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 766 00000100:00100000:1.0:1788845034.842014:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660289920:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.842210:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845034.842231:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ad637 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8da4 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.842252:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.842270:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.842277:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287ad637 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8da4 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.842292:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:1.0:1788845034.842532:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641610880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9206us (12520us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.842541:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1124 00000100:00100000:2.0:1788845034.845987:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.846026:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.846104:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641611136 00000100:00100000:3.0:1788845034.846114:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641611136 00000100:00100000:3.0:1788845034.846174:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1125 00000100:00100000:3.0:1788845034.846184:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641611136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.846223:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.846244:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad63e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8db2 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.846281:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.846318:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7d500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660290048:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.846411:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.846862:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641611264 00000100:00100000:1.0:1788845034.846867:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641611264 00000100:00100000:1.0:1788845034.846912:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 767 00000100:00100000:1.0:1788845034.846920:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641611264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.846931:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.846941:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad645 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8db9 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.846986:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660290176:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.847020:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.850479:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660290176:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845034.850524:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad645 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8db9 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.850551:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad645 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8db9 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.850564:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845034.850633:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641611264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3712us (4655us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.850641:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 767 00000100:00100000:3.0:1788845034.850968:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7d500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660290048:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845034.850984:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:3.0:1788845034.850996:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad63e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8db2 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.851011:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.851014:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845034.851019:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad63e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8db2 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.851031:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845034.851057:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641611136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4874us (5030us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.851066:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1125 00000100:00100000:1.0:1788845034.856004:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.856123:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641611392 00000100:00100000:3.0:1788845034.856132:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641611392 00000100:00100000:3.0:1788845034.856192:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1126 00000100:00100000:3.0:1788845034.856202:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641611392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.856239:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.856273:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287ad64c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8dc0 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.856310:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.856359:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7d500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660290304:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.856441:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.863318:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.863654:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7d500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660290304:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845034.863679:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:3.0:1788845034.863709:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287ad64c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dc0 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.863729:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.863732:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845034.863738:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287ad64c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dc0 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.863752:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fe00, rc 0) 00000100:00100000:3.0:1788845034.863824:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641611392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7622us (7829us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.863834:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1126 00000100:00100000:1.0:1788845034.863955:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641611520 00000100:00100000:1.0:1788845034.863963:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641611520 00000100:00100000:1.0:1788845034.864018:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 768 00000100:00100000:1.0:1788845034.864027:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641611520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.864042:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.864060:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad653 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8dc7 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.864115:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660290432:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.864186:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.866810:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660290432:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845034.866881:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad653 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dc7 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.867038:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad653 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dc7 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.867051:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:2.0:1788845034.867169:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641611520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3141us (3858us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.867178:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 768 00000100:00100000:2.0:1788845034.872328:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.872404:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641611648 00000100:00100000:1.0:1788845034.872415:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641611648 00000100:00100000:2.0:1788845034.874003:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 769 00000100:00100000:2.0:1788845034.874025:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641611648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.874046:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.874066:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8dce expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.874102:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.874111:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 8) 00010000:00010000:2.0:1788845034.874115:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:2.0:1788845034.874120:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1351680->1355775] (req 1351680->1355775) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.874156:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.874175:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 6 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.874191:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660290560:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845034.874415:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.877175:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660290560:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845034.877214:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1351680->1355775] (req 1351680->1355775) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.877236:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba400, rc 0) 00000100:00100000:2.0:1788845034.877298:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641611648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3282us (4978us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.877308:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 769 00000100:00100000:0.0:1788845034.878425:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.878518:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641611776 00000100:00100000:2.0:1788845034.878528:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641611776 00002000:00100000:2.0:1788845034.878575:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1310720->1368788) req@ffff94a02da49180 x1875739641611776/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.878586:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da49180 with x1875739641611776 ext(1310720->1368788) 00010000:00010000:2.0:1788845034.878596:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 7 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.878615:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 7 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845034.878624:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a02da49180 00000100:00100000:2.0:1788845034.878654:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 19 00000100:00100000:2.0:1788845034.878663:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641611776:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845034.888036:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641611776:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 9363us (9606us total) trans 4294967435 rc 0/0 00000100:00100000:2.0:1788845034.888050:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 19 00002000:00100000:2.0:1788845034.888063:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1310720->1368788) req@ffff94a02da49180 x1875739641611776/t4294967435(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/448 e 0 to 0 dl 1788845045 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.888073:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da49180 with x1875739641611776 ext(1310720->1368788) 00010000:00010000:2.0:1788845034.888083:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 9 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.888105:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 9 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845034.888113:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a02da49180 00000100:00100000:3.0:1788845034.890841:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.890953:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641611904 00000100:00100000:1.0:1788845034.890962:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641611904 00010000:00010000:1.0:1788845034.891000:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 9 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.891041:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 400 00000100:00100000:1.0:1788845034.891052:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641611904:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845034.891067:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845034.891090:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 9 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.891107:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 9 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.891131:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287ad61b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1310720->1372159) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d96 expref: 9 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.891163:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:1.0:1788845034.891167:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:1.0:1788845034.891175:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.891185:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.891223:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.891237:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008919880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660290688:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845034.891334:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.894758:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.894878:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008919880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660290688:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845034.894937:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845034.894993:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641611904:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3943us (4160us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.895021:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 400 00000100:00100000:2.0:1788845034.895422:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641612032 00000100:00100000:2.0:1788845034.895430:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641612032 00002000:00100000:2.0:1788845034.895474:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1351680->1355775) req@ffff94a00891aa00 x1875739641612032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.895485:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a00891aa00 with x1875739641612032 ext(1351680->1355775) 00002000:00010000:2.0:1788845034.895491:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a00891aa00 00000100:00100000:2.0:1788845034.895520:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 399 00000100:00100000:2.0:1788845034.895529:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641612032:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.895707:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641612032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 178us (959us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845034.895718:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 399 00002000:00100000:2.0:1788845034.895724:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1351680->1355775) req@ffff94a00891aa00 x1875739641612032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.895731:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a00891aa00 with x1875739641612032 ext(1351680->1355775) 00002000:00010000:2.0:1788845034.895735:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a00891aa00 00000100:00100000:1.0:1788845034.901218:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.901317:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641612160 00000100:00100000:2.0:1788845034.901329:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641612160 00002000:00100000:2.0:1788845034.901388:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1355776->1359871) req@ffff94a00891b480 x1875739641612160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.901400:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a00891b480 with x1875739641612160 ext(1355776->1359871) 00002000:00010000:2.0:1788845034.901407:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a00891b480 00000100:00100000:2.0:1788845034.901441:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 400 00000100:00100000:2.0:1788845034.901451:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641612160:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.901636:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641612160:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 185us (428us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845034.901644:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 400 00002000:00100000:2.0:1788845034.901651:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1355776->1359871) req@ffff94a00891b480 x1875739641612160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.901659:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a00891b480 with x1875739641612160 ext(1355776->1359871) 00002000:00010000:2.0:1788845034.901663:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a00891b480 00000100:00100000:1.0:1788845034.905172:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.905277:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641612288 00000100:00100000:0.0:1788845034.905289:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641612288 00002000:00100000:0.0:1788845034.905349:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1359872->1363967) req@ffff94a008919180 x1875739641612288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.905360:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a008919180 with x1875739641612288 ext(1359872->1363967) 00002000:00010000:0.0:1788845034.905365:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a008919180 00000100:00100000:0.0:1788845034.905400:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 401 00000100:00100000:0.0:1788845034.905409:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641612288:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.905636:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641612288:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 227us (477us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845034.905645:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 401 00002000:00100000:0.0:1788845034.905652:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1359872->1363967) req@ffff94a008919180 x1875739641612288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.905660:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a008919180 with x1875739641612288 ext(1359872->1363967) 00002000:00010000:0.0:1788845034.905664:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a008919180 00000100:00100000:3.0:1788845034.908826:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.909043:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641612416 00000100:00100000:1.0:1788845034.909054:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641612416 00002000:00100000:1.0:1788845034.909104:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1363968->1368063) req@ffff94a035b7ca80 x1875739641612416/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.909116:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7ca80 with x1875739641612416 ext(1363968->1368063) 00002000:00010000:1.0:1788845034.909121:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7ca80 00000100:00100000:1.0:1788845034.909154:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 402 00000100:00100000:1.0:1788845034.909164:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641612416:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845034.909398:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641612416:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (590us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845034.909406:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 402 00002000:00100000:1.0:1788845034.909413:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1363968->1368063) req@ffff94a035b7ca80 x1875739641612416/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845034.909422:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7ca80 with x1875739641612416 ext(1363968->1368063) 00002000:00010000:1.0:1788845034.909425:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7ca80 00000100:00100000:1.0:1788845034.913280:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.913373:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641612544 00000100:00100000:2.0:1788845034.913382:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641612544 00000100:00100000:2.0:1788845034.913433:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1127 00000100:00100000:2.0:1788845034.913442:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641612544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.913462:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.913482:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad661 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8dd5 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.913515:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.913549:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660290816:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.913627:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.916154:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660290816:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.916183:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845034.916201:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad661 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dd5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.916223:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.916227:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.916232:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad661 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dd5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.916249:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba800, rc 0) 00000100:00100000:1.0:1788845034.916308:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641612544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2865us (3036us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.916317:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1127 00000100:00100000:0.0:1788845034.919481:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.919584:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641612672 00000100:00100000:2.0:1788845034.919595:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641612672 00002000:00100000:2.0:1788845034.919642:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1368064->1372159) req@ffff94a02da4b800 x1875739641612672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.919653:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da4b800 with x1875739641612672 ext(1368064->1372159) 00002000:00010000:2.0:1788845034.919658:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da4b800 00000100:00100000:2.0:1788845034.919698:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 403 00000100:00100000:2.0:1788845034.919709:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641612672:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845034.919952:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641612672:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 244us (482us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845034.919960:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 403 00002000:00100000:2.0:1788845034.919966:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1368064->1372159) req@ffff94a02da4b800 x1875739641612672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:140/0 lens 488/4536 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.919975:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da4b800 with x1875739641612672 ext(1368064->1372159) 00002000:00010000:2.0:1788845034.919978:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da4b800 00000100:00100000:2.0:1788845034.923427:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845034.923521:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641612800 00000100:00100000:1.0:1788845034.923531:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641612800 00000100:00100000:1.0:1788845034.923590:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1128 00000100:00100000:1.0:1788845034.923601:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641612800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.923638:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.923659:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad668 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ddc expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.923707:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.923746:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660290944:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.923840:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.926151:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660290944:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.926179:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845034.926196:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad668 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ddc expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.926215:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.926218:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.926223:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad668 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ddc expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.926238:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845034.926309:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641612800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2710us (2893us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.926317:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1128 00000100:00100000:0.0:1788845034.929159:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.929244:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641612928 00000100:00100000:3.0:1788845034.929267:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641612928 00000100:00100000:3.0:1788845034.929338:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1129 00000100:00100000:3.0:1788845034.929349:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641612928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.929370:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.929389:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287ad66f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8de3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.929427:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.929474:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660291072:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.929562:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845034.932603:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660291072:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845034.932629:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:3.0:1788845034.932649:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287ad66f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8de3 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.932670:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845034.932673:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845034.932678:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287ad66f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8de3 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.932705:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fe00, rc 0) 00000100:00100000:3.0:1788845034.932763:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641612928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3416us (3614us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845034.932773:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1129 00000100:00100000:2.0:1788845034.936131:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845034.936238:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641613056 00000100:00100000:0.0:1788845034.936247:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641613056 00000100:00100000:0.0:1788845034.936311:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 770 00000100:00100000:0.0:1788845034.936321:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641613056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845034.936504:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845034.936523:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8dea expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.936572:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.936581:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:0.0:1788845034.936584:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:0.0:1788845034.936589:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1581056->1613823] (req 1581056->1613823) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.936624:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.936643:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 6 pid: 6684 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845034.936657:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660291200:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845034.936854:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.940986:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.941125:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641613184 00000100:00100000:2.0:1788845034.941135:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641613184 00010000:00010000:2.0:1788845034.941179:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 7 pid: 6684 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.941223:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 401 00000100:00100000:2.0:1788845034.941234:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641613184:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845034.941248:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845034.941580:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 7 pid: 6684 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.941600:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 7 pid: 6684 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.941626:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:2.0:1788845034.941631:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:2.0:1788845034.941639:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.941649:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.941699:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.941715:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660291328:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845034.941803:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06cc0 going to sleep for 11 seconds 00010000:00010000:3.0:1788845034.943775:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003f45a18b/0xb2187301287ad65a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dce expref: 7 pid: 6684 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.943830:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660291200:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845034.943861:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845034.943878:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772400, rc 0) 00000100:00100000:2.0:1788845034.944284:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641613056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7962us (8160us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845034.944293:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 770 00000100:00100000:3.0:1788845034.947592:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.947637:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.947666:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641613440 00000100:00100000:3.0:1788845034.947675:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641613440 00000100:00100000:3.0:1788845034.947735:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 771 00000100:00100000:3.0:1788845034.947744:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641613440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845034.947764:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845034.947782:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad67d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8df8 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845034.947850:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660291456:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845034.947948:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.948099:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641613312 00000100:00100000:2.0:1788845034.948104:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641613312 00000100:00100000:2.0:1788845034.948144:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1130 00000100:00100000:2.0:1788845034.948152:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641613312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.948162:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.948175:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ad684 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8df1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.948202:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.948225:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660291584:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.948268:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845034.950016:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660291328:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845034.950042:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845034.950096:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641613184:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8864us (9120us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845034.950107:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 401 00002000:00010000:0.0:1788845034.951510:0:6685:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1368789 -> 1612710 00000100:00100000:0.0:1788845034.951536:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660291456:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845034.951585:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad67d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8df8 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.951610:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000072ec1400/0xb2187301287ad67d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8df8 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.951623:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fe00, rc 0) 00000100:00100000:0.0:1788845034.951670:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641613440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3926us (4031us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.951678:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 771 00000100:00100000:0.0:1788845034.953585:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660291584:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845034.953610:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845034.953627:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ad684 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8df1 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.953645:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845034.953648:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845034.953653:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbac00/0xb2187301287ad684 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8df1 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845034.953667:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbac00, rc 0) 00000100:00100000:0.0:1788845034.953733:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641613312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5581us (6148us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845034.953742:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1130 00000100:00100000:3.0:1788845034.957065:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.957104:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.957276:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641613696 00000100:00100000:2.0:1788845034.957286:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641613696 00000100:00100000:2.0:1788845034.957351:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 772 00000100:00100000:2.0:1788845034.957361:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641613696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845034.957382:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845034.957399:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ad68b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e06 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845034.957455:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660291712:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845034.957533:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845034.957653:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641613568 00000100:00100000:1.0:1788845034.957657:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641613568 00000100:00100000:1.0:1788845034.957705:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1131 00000100:00100000:1.0:1788845034.957713:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641613568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845034.957749:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845034.957761:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad692 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8dff expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.957792:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845034.957827:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660291840:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845034.957855:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845034.960739:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660291712:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845034.960777:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ad68b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e06 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.960803:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f1b65633/0xb2187301287ad68b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e06 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845034.960827:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:2.0:1788845034.960926:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641613696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3523us (3826us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845034.960935:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 772 00000100:00100000:1.0:1788845034.961161:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660291840:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845034.961177:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845034.961188:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad692 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dff expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.961200:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845034.961203:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845034.961207:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad692 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dff expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845034.961217:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845034.961264:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641613568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3540us (4148us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845034.961273:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1131 00000100:00100000:3.0:1788845034.972194:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845034.972258:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845034.972358:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641613824 00000100:00100000:2.0:1788845034.972368:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641613824 00002000:00100000:2.0:1788845034.972418:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03592a300 x1875739641613824/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845034.972430:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03592a300 with x1875739641613824 00000100:00100000:2.0:1788845034.972471:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 756 00000100:00100000:2.0:1788845034.972481:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641613824:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845034.972786:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641613952 00000100:00100000:0.0:1788845034.972791:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641613952 00002000:00100000:0.0:1788845034.972840:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1581055) req@ffff94a035929f80 x1875739641613952/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/0 e 0 to 0 dl 1788845045 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845034.972851:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035929f80 with x1875739641613952 ext(1048576->1581055) 00002000:00010000:0.0:1788845034.972856:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035929f80 00000100:00100000:0.0:1788845034.972911:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 404 00000100:00100000:0.0:1788845034.972920:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641613952:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845035.022949:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641613824:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 50459us (50681us total) trans 0 rc 585728/585728 00000100:00100000:2.0:1788845035.022963:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 756 00002000:00100000:2.0:1788845035.022976:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03592a300 x1875739641613824/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 585728/585728 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.022991:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03592a300 with x1875739641613824 00000100:00100000:0.0:1788845035.025055:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641613952:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 52127us (52862us total) trans 0 rc 323584/323584 00000100:00100000:0.0:1788845035.025069:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 404 00002000:00100000:0.0:1788845035.025081:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1581055) req@ffff94a035929f80 x1875739641613952/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:140/0 lens 488/440 e 0 to 0 dl 1788845045 ref 1 fl Complete:/600/0 rc 323584/323584 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.025092:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035929f80 with x1875739641613952 ext(1048576->1581055) 00002000:00010000:0.0:1788845035.025097:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035929f80 00000100:00100000:0.0:1788845035.030512:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.030605:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641614080 00000100:00100000:3.0:1788845035.030616:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641614080 00000100:00100000:3.0:1788845035.030696:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1132 00000100:00100000:3.0:1788845035.030722:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641614080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.030744:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.030765:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad699 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e0d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.030804:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.030857:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660291968:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.030973:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00080000:1.0:1788845035.032128:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845035.037932:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000004:00010000:1.0:1788845035.038231:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845034 -> 1788845035 00000100:00100000:1.0:1788845035.038257:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660291968:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845035.038289:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845035.038310:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad699 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e0d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.038337:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.038340:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845035.038346:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287ad699 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e0d expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.038360:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e000, rc 0) 00000100:00100000:1.0:1788845035.038416:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641614080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7696us (7913us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.038425:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1132 00000100:00100000:2.0:1788845035.039182:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641614208 00000100:00100000:2.0:1788845035.039192:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641614208 00000100:00100000:2.0:1788845035.039256:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 773 00000100:00100000:2.0:1788845035.039266:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641614208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.039283:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.039299:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287ad6a0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e14 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.039367:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660292096:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.039435:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00000100:00080000:1.0:1788845035.039521:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1788845035.042718:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845034 -> 1788845035 00000100:00100000:2.0:1788845035.042749:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660292096:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845035.042785:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287ad6a0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e14 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.042819:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0c3bff8/0xb2187301287ad6a0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e14 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.042837:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb600, rc 0) 00000100:00100000:2.0:1788845035.043106:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641614208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3840us (5221us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.043116:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 773 00000100:00100000:2.0:1788845035.046450:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.046531:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.046595:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641614336 00000100:00100000:3.0:1788845035.046605:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641614336 00000100:00100000:3.0:1788845035.046660:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1133 00000100:00100000:3.0:1788845035.046671:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641614336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.046702:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.046723:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad6a7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e1b expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.046759:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.046801:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660292224:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.046915:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.049232:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641614464 00000100:00100000:0.0:1788845035.049246:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641614464 00000100:00100000:1.0:1788845035.049726:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 774 00000100:00100000:1.0:1788845035.049737:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641614464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.049755:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.049774:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad6ae lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e22 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.049845:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660292352:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.049955:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.052518:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660292224:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.052547:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.052568:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad6a7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e1b expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.052588:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.052591:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.052596:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287ad6a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e1b expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.052610:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ec00, rc 0) 00000100:00100000:2.0:1788845035.052674:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641614336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6005us (6143us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.052691:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1133 00000100:00100000:1.0:1788845035.055507:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660292352:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.055550:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad6ae lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e22 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.055598:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad6ae lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e22 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.055611:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845035.055672:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641614464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5937us (9232us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.055690:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 774 00000100:00100000:0.0:1788845035.059821:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.059873:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.059991:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641614720 00000100:00100000:1.0:1788845035.060001:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641614720 00000100:00100000:1.0:1788845035.060055:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 775 00000100:00100000:1.0:1788845035.060065:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641614720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.060084:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.060102:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ad6b5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e30 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.060164:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660292480:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.060264:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.060449:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641614592 00000100:00100000:2.0:1788845035.060454:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641614592 00000100:00100000:2.0:1788845035.060493:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1134 00000100:00100000:2.0:1788845035.060501:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641614592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.060512:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.060525:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ad6bc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e29 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.060553:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.060576:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358e680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660292608:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.060607:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.063602:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358e680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660292608:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.063629:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.063646:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ad6bc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e29 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.063665:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.063668:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.063697:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb600/0xb2187301287ad6bc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e29 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.063714:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb600, rc 0) 00000100:00100000:0.0:1788845035.063768:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641614592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3267us (3966us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.063777:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1134 00000100:00100000:1.0:1788845035.066249:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660292480:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.066290:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ad6b5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e30 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.066314:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ad6b5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e30 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.066348:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845035.066404:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641614720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6341us (6530us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.066413:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 775 00000100:00100000:2.0:1788845035.071788:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.071980:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641614848 00000100:00100000:1.0:1788845035.071990:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641614848 00000100:00100000:1.0:1788845035.072047:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1135 00000100:00100000:1.0:1788845035.072057:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641614848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.072080:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.072101:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e37 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.072139:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.072182:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660292736:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.072255:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.075052:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.075149:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641614976 00000100:00100000:0.0:1788845035.075158:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641614976 00000100:00100000:0.0:1788845035.075215:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 776 00000100:00100000:0.0:1788845035.075225:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641614976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.075243:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.075272:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ad6ca lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e3e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.075451:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f827b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660292864:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.075536:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.078665:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660292736:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845035.078713:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845035.078735:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6c3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e37 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.078756:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.078759:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845035.078780:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e37 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.078795:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845035.078872:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641614848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6815us (7093us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.078928:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1135 00000100:00100000:2.0:1788845035.079554:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f827b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660292864:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845035.079583:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ad6ca lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e3e expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.079602:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f995ea91/0xb2187301287ad6ca lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e3e expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.079614:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031e6a800, rc 0) 00000100:00100000:2.0:1788845035.079664:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641614976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4440us (4621us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.079672:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 776 00000100:00100000:0.0:1788845035.084796:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.084944:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641615104 00000100:00100000:1.0:1788845035.084955:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641615104 00000100:00100000:1.0:1788845035.085015:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1136 00000100:00100000:1.0:1788845035.085026:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641615104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.085048:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.085073:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e45 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.085106:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.085116:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.085151:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.085169:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 15 pid: 5839 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.085183:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660292992:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845035.085285:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.090249:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845035.090490:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845035.090536:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641615232 00000100:00100000:3.0:1788845035.090547:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641615232 00010000:00010000:3.0:1788845035.090610:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 16 pid: 5839 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.090668:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 402 00000100:00100000:3.0:1788845035.090679:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641615232:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845035.090703:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845035.090728:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 16 pid: 5839 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.090746:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 16 pid: 5839 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.090766:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.090772:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.090784:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.090794:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.090840:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.090856:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035928380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660293120:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845035.090951:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00080000:2.0:1788845035.091419:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788845035.091525:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad606 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d81 expref: 16 pid: 5839 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.091543:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660292992:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845035.091567:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.091580:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.091583:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845035.091586:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845035.091625:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641615104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6601us (6838us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.091633:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1136 00000100:00100000:0.0:1788845035.095820:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.095929:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641615360 00000100:00100000:1.0:1788845035.095939:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641615360 00002000:00100000:1.0:1788845035.095989:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824e00 x1875739641615360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.096001:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824e00 with x1875739641615360 00000100:00100000:1.0:1788845035.096042:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 757 00000100:00100000:1.0:1788845035.096053:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641615360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.096303:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641615360:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 252us (504us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845035.096311:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 757 00002000:00100000:1.0:1788845035.096319:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824e00 x1875739641615360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.096344:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824e00 with x1875739641615360 00000100:00100000:3.0:1788845035.096553:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035928380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660293120:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845035.096579:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845035.096617:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641615232:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5939us (6378us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845035.096626:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 402 00000100:00100000:0.0:1788845035.100319:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.100400:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641615488 00000100:00100000:1.0:1788845035.100407:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641615488 00002000:00100000:1.0:1788845035.100446:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824000 x1875739641615488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.100454:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824000 with x1875739641615488 00000100:00100000:1.0:1788845035.100484:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 758 00000100:00100000:1.0:1788845035.100491:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641615488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.100707:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641615488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 216us (396us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845035.100713:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 758 00002000:00100000:1.0:1788845035.100718:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824000 x1875739641615488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.100723:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824000 with x1875739641615488 00000100:00100000:0.0:1788845035.104457:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.104554:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641615616 00000100:00100000:2.0:1788845035.104564:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641615616 00000100:00100000:2.0:1788845035.104627:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1137 00000100:00100000:2.0:1788845035.104637:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641615616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.104658:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.104679:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287ad6d8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e4c expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.104719:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.104764:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660293248:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.104854:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00000004:00010000:2.0:1788845035.106824:0:11036:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845034 -> 1788845035 00000004:00010000:2.0:1788845035.106833:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845034 -> 1788845035 00000100:00100000:2.0:1788845035.106860:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660293248:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.106924:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.106943:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287ad6d8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e4c expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.106962:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.106965:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.106969:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb400/0xb2187301287ad6d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e4c expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.106986:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb400, rc 0) 00000100:00100000:2.0:1788845035.107030:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641615616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2394us (2583us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.107038:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1137 00000100:00100000:0.0:1788845035.110371:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.110523:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641615744 00000100:00100000:2.0:1788845035.110532:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641615744 00000100:00100000:2.0:1788845035.110592:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1138 00000100:00100000:2.0:1788845035.110601:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641615744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.110621:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.110641:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbae00/0xb2187301287ad6df lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e53 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.110690:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.110729:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660293376:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.110824:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.113815:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660293376:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.113846:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.113868:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbae00/0xb2187301287ad6df lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e53 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.113970:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.113975:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.113980:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbae00/0xb2187301287ad6df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e53 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.113998:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbae00, rc 0) 00000100:00100000:2.0:1788845035.114063:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641615744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3462us (3700us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.114072:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1138 00000100:00100000:3.0:1788845035.117594:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.117691:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641615872 00000100:00100000:2.0:1788845035.117700:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641615872 00000100:00100000:2.0:1788845035.117764:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 777 00000100:00100000:2.0:1788845035.117773:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641615872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.117794:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.117818:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ad6e6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e5a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.117876:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660293504:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.117978:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00002000:00010000:2.0:1788845035.120144:0:6684:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845034 -> 1788845035 00002000:00010000:2.0:1788845035.120153:0:6684:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845034 -> 1788845035 00000100:00100000:2.0:1788845035.120177:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660293504:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845035.120214:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ad6e6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e5a expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.120237:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f74571ae/0xb2187301287ad6e6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e5a expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.120251:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb400, rc 0) 00000100:00100000:2.0:1788845035.120323:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641615872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2550us (2738us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.120521:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 777 00000100:00100000:0.0:1788845035.123734:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.123820:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641616000 00000100:00100000:1.0:1788845035.123830:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641616000 00000100:00100000:1.0:1788845035.123914:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 778 00000100:00100000:1.0:1788845035.123925:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641616000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.123945:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.123963:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ad6ed lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e61 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.124024:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660293632:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.124119:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.127533:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660293632:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.127574:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ad6ed lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e61 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.127600:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ad6ed lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e61 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.127613:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845035.127677:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641616000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3753us (3951us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.127695:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 778 00000100:00100000:3.0:1788845035.131104:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.131206:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641616256 00000100:00100000:1.0:1788845035.131215:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641616256 00000100:00100000:1.0:1788845035.131280:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1139 00000100:00100000:1.0:1788845035.131290:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641616256:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845035.131351:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845035.131377:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ad629 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d9d expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.131396:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.131400:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845035.131593:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.131608:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.131636:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.131668:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.131698:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 14 pid: 11036 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.131713:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660293760:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845035.131795:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.134204:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660293760:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845035.134233:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.135387:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.136422:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641616384 00000100:00100000:1.0:1788845035.136434:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641616384 00010000:00010000:1.0:1788845035.136479:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 15 pid: 11036 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.136569:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 403 00000100:00100000:1.0:1788845035.136580:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641616384:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845035.136595:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845035.136603:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 15 pid: 11036 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.136670:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 15 pid: 11036 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.136703:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad5e3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8d5e expref: 15 pid: 11036 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.136720:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.136726:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.136734:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.136743:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.137063:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845035.137115:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641616384:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 536us (1732us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.137123:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 403 00010000:00010000:3.0:1788845035.138479:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.138492:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845035.138614:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967717 00000004:00010000:3.0:1788845035.138665:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:3.0:1788845035.138669:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:3.0:1788845035.138672:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845034 00000004:00010000:3.0:1788845035.138674:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:3.0:1788845035.138677:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:3.0:1788845035.139113:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003421f8c2 reply state 00000000446dbfc7 transno 4294967718 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.139125:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845035.139452:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641616256:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8159us (8353us total) trans 4294967718 rc 0/0 00000100:00100000:3.0:1788845035.139461:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1139 00000020:00080000:3.0:1788845035.139467:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788845035.144562:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845035.144716:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.144743:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.144751:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.144757:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845035.144762:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.144770:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.144787:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.144791:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845035.144795:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.144801:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad6f4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.145008:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641616512 00000100:00100000:1.0:1788845035.145018:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641616512 00000100:00100000:1.0:1788845035.145113:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 779 00000100:00100000:1.0:1788845035.145124:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641616512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.145143:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.145160:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e68 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.145204:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.145212:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:1.0:1788845035.145217:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 8) 00010000:00010000:1.0:1788845035.145222:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.145255:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.145273:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 8 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.145287:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660293888:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845035.145379:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.148084:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.148200:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641616640 00000100:00100000:0.0:1788845035.148208:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641616640 00002000:00100000:0.0:1788845035.148268:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1581056->1612709) req@ffff94a03592b800 x1875739641616640/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.148294:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03592b800 with x1875739641616640 ext(1581056->1612709) 00010000:00010000:0.0:1788845035.148305:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 9 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.148336:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 9 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845035.148346:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03592b800 00000100:00100000:0.0:1788845035.148373:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 20 00000100:00100000:0.0:1788845035.148383:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641616640:12345-192.168.201.29@tcp:4:fsx.0 00000100:00080000:0.0:1788845035.148514:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845035.150973:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641616640:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 2582us (2889us total) trans 4294967436 rc 0/0 00000100:00100000:0.0:1788845035.150987:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 20 00002000:00100000:0.0:1788845035.150999:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1581056->1612709) req@ffff94a03592b800 x1875739641616640/t4294967436(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/448 e 0 to 0 dl 1788845046 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.151009:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03592b800 with x1875739641616640 ext(1581056->1612709) 00010000:00010000:0.0:1788845035.151019:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 11 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.151040:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 11 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845035.151048:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03592b800 00000100:00100000:1.0:1788845035.151305:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660293888:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845035.151348:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.151363:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845035.151392:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641616512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6269us (6839us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.151410:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 779 00000100:00100000:0.0:1788845035.155916:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.156004:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641616768 00000100:00100000:2.0:1788845035.156015:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641616768 00010000:00010000:2.0:1788845035.156056:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 11 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.156094:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 404 00000100:00100000:2.0:1788845035.156104:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d49c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641616768:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845035.156121:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845035.156143:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 11 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.156161:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 11 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.156184:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287ad676 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1613823) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8dea expref: 11 pid: 6682 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.156202:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 8) 00010000:00010000:2.0:1788845035.156206:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 8) 00010000:00010000:2.0:1788845035.156214:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.156224:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.156261:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.156275:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660294016:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845035.156365:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a003d06840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.159408:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.159491:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660294016:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845035.159514:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845035.159566:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d49c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641616768:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3463us (3687us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.159576:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 404 00000100:00100000:3.0:1788845035.159663:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641616896 00000100:00100000:3.0:1788845035.159669:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641616896 00002000:00010000:3.0:1788845035.159707:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1953551->18446744073709551615) 00002000:00100000:3.0:1788845035.159711:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f825f80 with x1875739641616896 ext(1953551->18446744073709551615) 00002000:00010000:3.0:1788845035.159721:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f825f80. 00000100:00100000:3.0:1788845035.159748:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 405 00000100:00100000:3.0:1788845035.159756:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641616896:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845035.160039:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1612710 -> 1953551 00002000:00010000:3.0:1788845035.160043:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00002000:00010000:3.0:1788845035.160045:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845035 -> 1788845034 00002000:00010000:3.0:1788845035.160048:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00002000:00010000:3.0:1788845035.160050:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 81 -> 81 00000100:00100000:3.0:1788845035.160076:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641616896:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 321us (677us total) trans 4294967437 rc 0/0 00000100:00100000:3.0:1788845035.160084:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 405 00002000:00010000:3.0:1788845035.160087:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1953551->18446744073709551615) 00002000:00100000:3.0:1788845035.160090:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f825f80 with x1875739641616896 ext(1953551->18446744073709551615) 00002000:00010000:3.0:1788845035.160094:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f825f80. 00000100:00100000:0.0:1788845035.169045:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.169093:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.169193:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641617024 00000100:00100000:2.0:1788845035.169204:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641617024 00002000:00100000:2.0:1788845035.169256:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1953791) req@ffff94a03f825180 x1875739641617024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.169267:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f825180 with x1875739641617024 ext(1048576->1953791) 00002000:00010000:2.0:1788845035.169273:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f825180 00000100:00100000:2.0:1788845035.169307:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 406 00000100:00100000:2.0:1788845035.169318:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641617024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.173123:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641617152 00000100:00100000:1.0:1788845035.173139:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641617152 00002000:00100000:1.0:1788845035.173190:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826300 x1875739641617152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.173202:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826300 with x1875739641617152 00000100:00100000:1.0:1788845035.173249:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 759 00000100:00100000:1.0:1788845035.173258:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641617152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.180052:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641617152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 6784us (11007us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.180067:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 759 00002000:00100000:1.0:1788845035.180079:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f826300 x1875739641617152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/440 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.180088:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f826300 with x1875739641617152 00000100:00100000:2.0:1788845035.183853:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641617024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14526us (14748us total) trans 0 rc 905216/905216 00000100:00100000:2.0:1788845035.183868:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 406 00002000:00100000:2.0:1788845035.183880:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1953791) req@ffff94a03f825180 x1875739641617024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/440 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 905216/905216 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.183927:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f825180 with x1875739641617024 ext(1048576->1953791) 00002000:00010000:2.0:1788845035.183933:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f825180 00000100:00100000:0.0:1788845035.192419:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.192519:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641617280 00000100:00100000:1.0:1788845035.192529:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641617280 00002000:00100000:1.0:1788845035.192580:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1892352->1953550) req@ffff94a03f824380 x1875739641617280/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.192592:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f824380 with x1875739641617280 ext(1892352->1953550) 00002000:00010000:1.0:1788845035.192598:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f824380 00000100:00100000:1.0:1788845035.192631:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 407 00000100:00100000:1.0:1788845035.192641:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641617280:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845035.197046:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641617280:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4396us (4626us total) trans 4294967438 rc 0/0 00000100:00100000:1.0:1788845035.197074:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 407 00002000:00100000:1.0:1788845035.197087:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1892352->1953550) req@ffff94a03f824380 x1875739641617280/t4294967438(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/448 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.197097:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f824380 with x1875739641617280 ext(1892352->1953550) 00002000:00010000:1.0:1788845035.197103:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f824380 00000100:00100000:0.0:1788845035.200422:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.200566:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641617408 00000100:00100000:2.0:1788845035.200579:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641617408 00000100:00100000:2.0:1788845035.200638:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 780 00000100:00100000:2.0:1788845035.200648:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641617408:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845035.234966:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967436 is committed 00000001:00080000:1.0:1788845035.234989:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967438 is committed 00000100:00100000:2.0:1788845035.235319:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641617408:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 34666us (34901us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.235340:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 780 00000100:00100000:3.0:1788845035.238169:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.238309:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641617536 00000100:00100000:1.0:1788845035.238319:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641617536 00000100:00100000:1.0:1788845035.238449:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1140 00000100:00100000:1.0:1788845035.238458:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641617536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.238478:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.238498:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad702 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e6f expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.238642:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad709 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.238657:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad709 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.238667:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad709 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845035.238763:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1953551 00000004:00200000:1.0:1788845035.238768:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845035.238788:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad709 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.238840:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ad702 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e6f expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.238857:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad709 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e6f expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.238867:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.238869:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845035.238873:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845035.239247:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641617536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 789us (1088us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.239264:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1140 00000100:00100000:3.0:1788845035.242027:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.242111:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.242184:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641617664 00000100:00100000:0.0:1788845035.242195:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641617664 00000100:00100000:0.0:1788845035.242254:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1141 00000100:00100000:0.0:1788845035.242264:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641617664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.242287:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.242309:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660600/0xb2187301287ad710 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e76 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.242355:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.242398:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660294144:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.242471:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000004:00010000:0.0:1788845035.244509:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845034 -> 1788845035 00000100:00100000:0.0:1788845035.244541:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660294144:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.244564:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.244581:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660600/0xb2187301287ad710 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e76 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.244602:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.244606:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.244626:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660600/0xb2187301287ad710 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e76 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.244642:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a001660600, rc 0) 00000100:00100000:0.0:1788845035.244712:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641617664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2447us (2598us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.244720:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1141 00000100:00100000:2.0:1788845035.246104:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641617792 00000100:00100000:2.0:1788845035.246114:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641617792 00000100:00100000:2.0:1788845035.246173:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 781 00000100:00100000:2.0:1788845035.246183:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641617792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.246201:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.246219:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000016e3697b/0xb2187301287ad717 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e7d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.246276:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660294272:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.246726:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845035.249656:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845034 -> 1788845035 00002000:00010000:2.0:1788845035.249665:0:6684:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 81 -> 317 00000100:00100000:2.0:1788845035.249700:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660294272:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845035.249738:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000016e3697b/0xb2187301287ad717 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e7d expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.249763:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000016e3697b/0xb2187301287ad717 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e7d expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.249778:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba200, rc 0) 00000100:00100000:2.0:1788845035.249857:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641617792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3674us (7838us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.249865:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 781 00000100:00100000:3.0:1788845035.256063:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.256102:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.256188:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641618048 00000100:00100000:1.0:1788845035.256198:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641618048 00000100:00100000:1.0:1788845035.256264:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 782 00000100:00100000:1.0:1788845035.256273:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641618048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.256291:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.256307:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad71e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e8b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.256374:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660294400:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.256454:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.256917:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641617920 00000100:00100000:2.0:1788845035.256923:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641617920 00000100:00100000:2.0:1788845035.256962:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1142 00000100:00100000:2.0:1788845035.256971:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641617920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.256982:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.256997:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad725 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e84 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.257026:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.257054:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660294528:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.257099:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.259972:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660294400:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.260014:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad71e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e8b expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.260063:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ad71e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e8b expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.260078:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:1.0:1788845035.260139:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641618048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3866us (4035us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.260148:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 782 00000100:00100000:2.0:1788845035.262161:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660294528:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.262188:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.262207:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad725 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e84 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.262227:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.262230:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.262235:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbaa00/0xb2187301287ad725 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e84 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.262248:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbaa00, rc 0) 00000100:00100000:2.0:1788845035.262525:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641617920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5544us (6461us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.262534:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1142 00000100:00100000:0.0:1788845035.266201:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.266555:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641618176 00000100:00100000:3.0:1788845035.266566:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641618176 00000100:00100000:3.0:1788845035.266657:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1143 00000100:00100000:3.0:1788845035.266667:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641618176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.266699:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.266720:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad72c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e92 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.266818:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.266852:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.266863:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845035.266928:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1953551 00000004:00200000:3.0:1788845035.266934:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845035.266953:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.266980:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad72c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.266999:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.267007:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.267010:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845035.267014:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845035.267077:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641618176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 412us (886us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845035.267085:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1143 00000100:00100000:0.0:1788845035.273033:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.273147:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641618304 00000100:00100000:1.0:1788845035.273158:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641618304 00002000:00100000:1.0:1788845035.273205:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f827b80 x1875739641618304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 504/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.273218:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f827b80 with x1875739641618304 00000100:00100000:1.0:1788845035.273265:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 760 00000100:00100000:1.0:1788845035.273275:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641618304:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.282070:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641618304:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8785us (9036us total) trans 0 rc 520192/520192 00000100:00100000:1.0:1788845035.282086:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 760 00002000:00100000:1.0:1788845035.282101:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f827b80 x1875739641618304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 504/440 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 520192/520192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.282119:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f827b80 with x1875739641618304 00000100:00100000:1.0:1788845035.285598:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.285695:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.285761:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641618560 00000100:00100000:3.0:1788845035.285762:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641618432 00000100:00100000:2.0:1788845035.285773:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641618560 00000100:00100000:3.0:1788845035.285843:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641618432 00000100:00100000:2.0:1788845035.285923:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 783 00000100:00100000:2.0:1788845035.285933:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641618560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.285948:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.285962:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005a92886d/0xb2187301287ad73a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ea0 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.285993:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1144 00000100:00100000:3.0:1788845035.286006:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641618432:12345-192.168.201.29@tcp:101:fsx.0 00000100:00100000:2.0:1788845035.286012:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660294656:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845035.286018:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.286045:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad741 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8e99 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.286079:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00010000:00010000:3.0:1788845035.286080:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.286108:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660294784:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.286137:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.288914:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660294656:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.288979:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005a92886d/0xb2187301287ad73a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ea0 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.289004:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005a92886d/0xb2187301287ad73a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ea0 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.289018:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbaa00, rc 0) 00000100:00100000:1.0:1788845035.289077:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641618560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3146us (3383us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.289086:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 783 00000100:00100000:1.0:1788845035.290352:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660294784:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845035.290375:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845035.290391:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad741 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e99 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.290407:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.290411:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845035.290416:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287ad741 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e99 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.290430:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e800, rc 0) 00000100:00100000:1.0:1788845035.290477:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641618432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4475us (4889us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.290485:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1144 00000100:00100000:1.0:1788845035.293731:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.293770:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.294060:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641618816 00000100:00100000:3.0:1788845035.294070:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641618816 00000100:00100000:3.0:1788845035.294125:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 784 00000100:00100000:3.0:1788845035.294136:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641618816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.294157:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.294175:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ad748 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8eae expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.294232:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641618688 00000100:00100000:3.0:1788845035.294235:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660294912:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.294238:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641618688 00000100:00100000:3.0:1788845035.294322:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.294596:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1145 00000100:00100000:0.0:1788845035.294606:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641618688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.294618:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.294632:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad74f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ea7 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.294663:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.294694:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660295040:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.294724:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.304047:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660294912:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845035.304087:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ad748 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8eae expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.304112:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ad748 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8eae expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.304126:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:3.0:1788845035.304201:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641618816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10066us (10479us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845035.304210:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 784 00000100:00100000:0.0:1788845035.304327:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660295040:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.304353:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.304365:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad74f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ea7 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.304393:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.304396:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.304402:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad74f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ea7 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.304414:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f600, rc 0) 00000100:00100000:0.0:1788845035.304437:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641618688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9834us (10666us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.304446:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1145 00000100:00100000:2.0:1788845035.312715:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.315536:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641618944 00000100:00100000:0.0:1788845035.315549:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641618944 00000100:00100000:2.0:1788845035.315699:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 761 00000100:00100000:2.0:1788845035.315710:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641618944:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845035.316150:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:2.0:1788845035.316155:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.316157:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.316159:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.316161:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:2.0:1788845035.316271:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641618944:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 561us (3563us total) trans 4294967719 rc 0/0 00000100:00100000:2.0:1788845035.316278:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 761 00000100:00100000:2.0:1788845035.320234:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.320486:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.320607:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641619072 00000100:00100000:1.0:1788845035.320617:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641619072 00000100:00100000:1.0:1788845035.320677:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1146 00000100:00100000:1.0:1788845035.320695:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641619072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.320716:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.320736:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad756 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8eb5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.321130:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.321179:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660295168:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.321276:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.323702:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660295168:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.323731:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.323750:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad756 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8eb5 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.323777:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.323781:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.323786:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad756 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8eb5 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.323801:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:2.0:1788845035.323876:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641619072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3181us (3651us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.324018:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1146 00000100:00100000:0.0:1788845035.325189:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641619200 00000100:00100000:0.0:1788845035.325199:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641619200 00000100:00100000:0.0:1788845035.325252:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 785 00000100:00100000:0.0:1788845035.325268:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641619200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.325285:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.325300:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000518aaa22/0xb2187301287ad75d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ebc expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.325354:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660295296:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.325584:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.328704:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660295296:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.328750:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000518aaa22/0xb2187301287ad75d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ebc expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.328799:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000518aaa22/0xb2187301287ad75d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ebc expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.328824:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f600, rc 0) 00000100:00100000:0.0:1788845035.329250:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641619200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3981us (8763us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.329274:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 785 00000100:00100000:0.0:1788845035.338461:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.338497:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641619456 00000100:00100000:0.0:1788845035.338507:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641619456 00000100:00100000:0.0:1788845035.338556:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 786 00000100:00100000:0.0:1788845035.338566:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641619456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.338585:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.338603:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad764 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8eca expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.338673:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660295424:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.338768:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.338832:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.338857:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641619328 00000100:00100000:0.0:1788845035.338861:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641619328 00000100:00100000:0.0:1788845035.338921:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1147 00000100:00100000:0.0:1788845035.338929:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641619328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.338941:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.338968:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287ad76b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ec3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.338999:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.339022:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660295552:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.339052:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.341719:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660295552:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845035.341749:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:3.0:1788845035.341768:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287ad76b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ec3 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.341794:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.341798:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845035.341815:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287ad76b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ec3 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.341833:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:3.0:1788845035.341915:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641619328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2985us (3081us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845035.341923:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1147 00000100:00100000:0.0:1788845035.342024:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660295424:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.342043:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad764 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8eca expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.342060:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad764 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8eca expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.342069:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:0.0:1788845035.342091:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641619456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3527us (3642us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.342098:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 786 00000100:00100000:0.0:1788845035.346867:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.347199:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641619584 00000100:00100000:2.0:1788845035.347209:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641619584 00000100:00100000:2.0:1788845035.347279:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1148 00000100:00100000:2.0:1788845035.347289:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641619584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.347310:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.347345:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad772 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ed1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.347386:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.347426:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660295680:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.347510:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.349981:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.350309:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660295680:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845035.350346:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:3.0:1788845035.350367:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad772 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ed1 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.350390:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.350394:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845035.350399:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba400/0xb2187301287ad772 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ed1 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.350441:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba400, rc 0) 00000100:00100000:3.0:1788845035.350513:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641619584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3224us (3655us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845035.350523:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1148 00000100:00100000:2.0:1788845035.351184:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641619712 00000100:00100000:2.0:1788845035.351192:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641619712 00000100:00100000:2.0:1788845035.351246:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 787 00000100:00100000:2.0:1788845035.351266:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641619712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.351286:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.351305:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad779 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ed8 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.351370:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660295808:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.351441:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.355159:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660295808:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.355203:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad779 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ed8 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.355229:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000930de83e/0xb2187301287ad779 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ed8 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.355274:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbac00, rc 0) 00000100:00100000:0.0:1788845035.355343:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641619712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4087us (5372us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.355351:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 787 00000100:00100000:3.0:1788845035.358139:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.358178:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.358277:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641619840 00000100:00100000:1.0:1788845035.358287:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641619840 00000100:00100000:1.0:1788845035.358349:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1149 00000100:00100000:1.0:1788845035.358359:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641619840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.358382:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.358402:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ad780 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8edf expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.358439:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.358480:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660295936:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.358566:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.358954:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641619968 00000100:00100000:0.0:1788845035.358959:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641619968 00000100:00100000:0.0:1788845035.358997:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 788 00000100:00100000:0.0:1788845035.359004:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641619968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.359016:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.359027:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad787 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ee6 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.359060:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660296064:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.359112:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.363235:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660296064:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.363292:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad787 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ee6 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.363316:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad787 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ee6 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.363337:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:0.0:1788845035.363542:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641619968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4535us (5410us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.363568:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 788 00000100:00100000:0.0:1788845035.364381:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660295936:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.364402:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.364418:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ad780 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8edf expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.364436:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.364439:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.364444:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ad780 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8edf expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.364457:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:0.0:1788845035.364507:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641619840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6150us (6327us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.364514:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1149 00000100:00100000:2.0:1788845035.370604:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.370698:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.370841:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641620096 00000100:00100000:3.0:1788845035.370851:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641620096 00000100:00100000:3.0:1788845035.370934:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1150 00000100:00100000:3.0:1788845035.370945:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641620096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.370967:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.370987:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad78e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8eed expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.371024:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.371063:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660296192:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.371141:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.373706:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660296192:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.373733:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.373754:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad78e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8eed expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.373777:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.373780:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.373785:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fc00/0xb2187301287ad78e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8eed expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.373801:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fc00, rc 0) 00000100:00100000:0.0:1788845035.373952:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641620096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3008us (3253us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.373961:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1150 00000100:00100000:1.0:1788845035.374127:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641620224 00000100:00100000:1.0:1788845035.374135:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641620224 00000100:00100000:1.0:1788845035.374274:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 789 00000100:00100000:1.0:1788845035.374284:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641620224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.374300:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.374317:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad795 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ef4 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.374378:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660296320:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.374428:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a006510f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.376499:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660296320:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.376543:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad795 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ef4 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.376565:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad795 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ef4 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.376579:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:1.0:1788845035.376637:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641620224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2354us (6041us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.376645:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 789 00000100:00100000:2.0:1788845035.379991:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.380089:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641620352 00000100:00100000:3.0:1788845035.380098:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641620352 00000100:00100000:3.0:1788845035.380163:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1151 00000100:00100000:3.0:1788845035.380173:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641620352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.380197:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.380218:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8efb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.380279:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.380291:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.380333:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.380352:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 16 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.380367:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660296448:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845035.380445:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.383496:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.383856:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660296448:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845035.383910:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.383931:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.383935:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.383939:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.383950:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:0.0:1788845035.384015:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641620352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3843us (4033us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845035.384024:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1151 00000100:00100000:2.0:1788845035.385208:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641620480 00000100:00100000:2.0:1788845035.385222:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641620480 00002000:00100000:2.0:1788845035.385487:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d3800 x1875739641620480/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 504/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.385498:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d3800 with x1875739641620480 00010000:00010000:2.0:1788845035.385507:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 17 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.385527:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 17 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845035.385536:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a0345d3800 00000100:00100000:2.0:1788845035.385610:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 26 00000100:00100000:2.0:1788845035.385621:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641620480:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845035.389745:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:2.0:1788845035.389754:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.389757:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.389761:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.389764:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:2.0:1788845035.390089:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641620480:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4464us (6597us total) trans 4294967720 rc 0/0 00000100:00100000:2.0:1788845035.390100:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 26 00002000:00100000:2.0:1788845035.390114:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d3800 x1875739641620480/t4294967720(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 504/448 e 0 to 0 dl 1788845046 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.390125:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d3800 with x1875739641620480 00010000:00010000:2.0:1788845035.390134:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 19 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.390154:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 19 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845035.390162:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a0345d3800 00000100:00100000:2.0:1788845035.395527:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.395823:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641620608 00000100:00100000:0.0:1788845035.395833:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641620608 00010000:00010000:0.0:1788845035.395876:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 19 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.396016:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 405 00000100:00100000:0.0:1788845035.396026:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641620608:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845035.396042:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845035.396069:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 19 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.396105:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 19 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.396129:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad6d1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e45 expref: 19 pid: 5840 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.396146:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.396153:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.396165:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.396175:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.396212:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.396227:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660296576:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845035.396334:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.400018:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.400500:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660296576:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845035.400525:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845035.400574:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641620608:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4549us (5056us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845035.400602:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 405 00000100:00100000:3.0:1788845035.400950:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641620736 00000100:00100000:3.0:1788845035.400959:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641620736 00002000:00100000:3.0:1788845035.401003:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ce00 x1875739641620736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.401014:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ce00 with x1875739641620736 00000100:00100000:3.0:1788845035.401048:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 762 00000100:00100000:3.0:1788845035.401057:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641620736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845035.401275:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641620736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 218us (1266us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845035.401282:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 762 00002000:00100000:3.0:1788845035.401295:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251ce00 x1875739641620736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.401304:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251ce00 with x1875739641620736 00000100:00100000:2.0:1788845035.404602:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.404867:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641620864 00000100:00100000:1.0:1788845035.404878:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641620864 00002000:00100000:1.0:1788845035.404962:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251d180 x1875739641620864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.404974:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251d180 with x1875739641620864 00000100:00100000:1.0:1788845035.405026:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 763 00000100:00100000:1.0:1788845035.405035:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641620864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.405297:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641620864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (705us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845035.405306:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 763 00002000:00100000:1.0:1788845035.405313:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251d180 x1875739641620864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.405320:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251d180 with x1875739641620864 00000100:00100000:3.0:1788845035.410730:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.410992:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641620992 00000100:00100000:1.0:1788845035.411002:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641620992 00002000:00100000:1.0:1788845035.411054:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d0e00 x1875739641620992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.411064:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d0e00 with x1875739641620992 00000100:00100000:1.0:1788845035.411106:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 764 00000100:00100000:1.0:1788845035.411117:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641620992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.411414:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641620992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 297us (691us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845035.411422:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 764 00002000:00100000:1.0:1788845035.411429:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d0e00 x1875739641620992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.411436:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d0e00 with x1875739641620992 00000100:00100000:3.0:1788845035.414692:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.414851:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641621120 00000100:00100000:1.0:1788845035.414861:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641621120 00002000:00100000:1.0:1788845035.414948:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d2d80 x1875739641621120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.414979:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d2d80 with x1875739641621120 00000100:00100000:1.0:1788845035.415178:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 765 00000100:00100000:1.0:1788845035.415189:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641621120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.415396:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641621120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 208us (724us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845035.415405:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 765 00002000:00100000:1.0:1788845035.415412:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d2d80 x1875739641621120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.415420:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d2d80 with x1875739641621120 00000100:00100000:3.0:1788845035.422283:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.422558:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641621248 00000100:00100000:1.0:1788845035.422569:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641621248 00002000:00100000:1.0:1788845035.422620:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d1880 x1875739641621248/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.422632:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d1880 with x1875739641621248 00000100:00100000:1.0:1788845035.422684:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 766 00000100:00100000:1.0:1788845035.422695:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641621248:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845035.424345:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641621248:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1639us (2063us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845035.424359:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 766 00002000:00100000:0.0:1788845035.424372:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d1880 x1875739641621248/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.424382:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d1880 with x1875739641621248 00000100:00100000:3.0:1788845035.427494:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.427612:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641621376 00000100:00100000:2.0:1788845035.427622:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641621376 00002000:00100000:2.0:1788845035.427671:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d3100 x1875739641621376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.427682:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d3100 with x1875739641621376 00000100:00100000:2.0:1788845035.427730:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 767 00000100:00100000:2.0:1788845035.427741:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641621376:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845035.428023:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641621376:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 282us (537us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845035.428032:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 767 00002000:00100000:2.0:1788845035.428039:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d3100 x1875739641621376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.428047:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d3100 with x1875739641621376 00000100:00100000:3.0:1788845035.432540:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.432772:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641621504 00000100:00100000:2.0:1788845035.432782:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641621504 00002000:00100000:2.0:1788845035.432840:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d3480 x1875739641621504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.432852:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d3480 with x1875739641621504 00000100:00100000:2.0:1788845035.432926:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 768 00000100:00100000:2.0:1788845035.432937:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641621504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845035.433153:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641621504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 217us (623us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845035.433162:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 768 00002000:00100000:2.0:1788845035.433169:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d3480 x1875739641621504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.433177:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d3480 with x1875739641621504 00000100:00100000:0.0:1788845035.436515:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.436791:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641621632 00000100:00100000:2.0:1788845035.436809:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641621632 00002000:00100000:2.0:1788845035.436855:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9c00 x1875739641621632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.436865:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9c00 with x1875739641621632 00000100:00100000:2.0:1788845035.436935:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 769 00000100:00100000:2.0:1788845035.436945:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641621632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845035.437317:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641621632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 372us (810us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845035.437334:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 769 00002000:00100000:2.0:1788845035.437342:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9c00 x1875739641621632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.437350:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9c00 with x1875739641621632 00000100:00100000:2.0:1788845035.444935:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.445026:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641621760 00000100:00100000:0.0:1788845035.445036:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641621760 00002000:00100000:0.0:1788845035.445086:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251d880 x1875739641621760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.445099:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251d880 with x1875739641621760 00000100:00100000:0.0:1788845035.445175:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 770 00000100:00100000:0.0:1788845035.445185:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641621760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845035.445410:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641621760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 225us (484us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845035.445418:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 770 00002000:00100000:0.0:1788845035.445426:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251d880 x1875739641621760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.445435:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251d880 with x1875739641621760 00000100:00100000:2.0:1788845035.448769:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.448912:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641621888 00000100:00100000:1.0:1788845035.448922:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641621888 00002000:00100000:1.0:1788845035.448969:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5df100 x1875739641621888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.448980:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5df100 with x1875739641621888 00000100:00100000:1.0:1788845035.449014:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 771 00000100:00100000:1.0:1788845035.449024:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641621888:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.449250:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641621888:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 226us (490us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845035.449259:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 771 00002000:00100000:1.0:1788845035.449266:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5df100 x1875739641621888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.449275:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5df100 with x1875739641621888 00000100:00100000:1.0:1788845035.452574:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.452937:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641622016 00000100:00100000:3.0:1788845035.452947:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641622016 00002000:00100000:3.0:1788845035.452996:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a00891a300 x1875739641622016/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.453007:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a00891a300 with x1875739641622016 00000100:00100000:3.0:1788845035.453045:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 772 00000100:00100000:3.0:1788845035.453055:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641622016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845035.453275:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641622016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 221us (711us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845035.453284:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 772 00002000:00100000:3.0:1788845035.453291:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a00891a300 x1875739641622016/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.453300:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a00891a300 with x1875739641622016 00000100:00100000:3.0:1788845035.458799:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.458952:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641622144 00000100:00100000:0.0:1788845035.458962:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641622144 00002000:00100000:0.0:1788845035.459013:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d2a00 x1875739641622144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.459039:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d2a00 with x1875739641622144 00000100:00100000:0.0:1788845035.459082:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 773 00000100:00100000:0.0:1788845035.459092:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641622144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845035.459314:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641622144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 222us (532us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845035.459330:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 773 00002000:00100000:0.0:1788845035.459337:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d2a00 x1875739641622144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.459345:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d2a00 with x1875739641622144 00000100:00100000:2.0:1788845035.463810:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.463937:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641622272 00000100:00100000:3.0:1788845035.463948:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641622272 00002000:00100000:3.0:1788845035.463996:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5ddc00 x1875739641622272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.464028:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5ddc00 with x1875739641622272 00000100:00100000:3.0:1788845035.464072:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 774 00000100:00100000:3.0:1788845035.464083:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641622272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845035.464304:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641622272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 221us (520us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845035.464312:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 774 00002000:00100000:3.0:1788845035.464329:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5ddc00 x1875739641622272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.464337:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5ddc00 with x1875739641622272 00000100:00100000:1.0:1788845035.467642:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.467750:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641622400 00000100:00100000:0.0:1788845035.467759:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641622400 00002000:00100000:0.0:1788845035.467814:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d6d80 x1875739641622400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.467826:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d6d80 with x1875739641622400 00000100:00100000:0.0:1788845035.467864:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 775 00000100:00100000:0.0:1788845035.467873:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641622400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845035.468124:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641622400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 251us (489us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845035.468132:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 775 00002000:00100000:0.0:1788845035.468138:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d6d80 x1875739641622400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.468147:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d6d80 with x1875739641622400 00000100:00100000:2.0:1788845035.471716:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.471816:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641622528 00000100:00100000:3.0:1788845035.471827:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641622528 00002000:00100000:3.0:1788845035.471877:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dfb80 x1875739641622528/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.471923:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dfb80 with x1875739641622528 00000100:00100000:3.0:1788845035.471971:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 776 00000100:00100000:3.0:1788845035.471980:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641622528:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845035.480688:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:1.0:1788845035.480697:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:1.0:1788845035.480699:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:1.0:1788845035.480701:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:1.0:1788845035.480703:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:1.0:1788845035.480781:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641622528:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 8797us (9071us total) trans 4294967721 rc 0/0 00000100:00100000:1.0:1788845035.480793:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 776 00002000:00100000:1.0:1788845035.480815:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dfb80 x1875739641622528/t4294967721(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/448 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.480824:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dfb80 with x1875739641622528 00000100:00100000:0.0:1788845035.483855:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.484087:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641622656 00000100:00100000:2.0:1788845035.484099:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641622656 00000100:00100000:2.0:1788845035.484157:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1152 00000100:00100000:2.0:1788845035.484167:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641622656:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845035.512015:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967715 is committed 00000001:00080000:1.0:1788845035.512130:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967721 is committed 00000001:00080000:1.0:1788845035.512140:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967717 is committed 00000001:00080000:1.0:1788845035.512148:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967720 is committed 00000100:00100000:0.0:1788845035.512399:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641622656:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 28227us (28546us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845035.512412:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1152 00000100:00100000:3.0:1788845035.515378:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.515458:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641622784 00000100:00100000:2.0:1788845035.515469:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641622784 00000100:00100000:2.0:1788845035.515534:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1153 00000100:00100000:2.0:1788845035.515545:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641622784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.515570:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.515594:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad7a3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f02 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.515632:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.515674:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383e300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660296704:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.516074:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000004:00010000:2.0:1788845035.518132:0:11036:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 518 -> 776 00000100:00100000:2.0:1788845035.518164:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383e300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660296704:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.518187:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.518206:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad7a3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f02 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.518225:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.518228:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.518233:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba800/0xb2187301287ad7a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f02 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.518248:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba800, rc 0) 00000100:00100000:2.0:1788845035.518292:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641622784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2749us (2923us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.518300:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1153 00000100:00100000:0.0:1788845035.521562:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.521655:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641622912 00000100:00100000:3.0:1788845035.521665:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641622912 00000100:00100000:3.0:1788845035.521828:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1154 00000100:00100000:3.0:1788845035.521838:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641622912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.521860:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.521880:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad7aa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f09 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.522047:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.522089:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660296832:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.522194:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.527152:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660296832:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.527182:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.527202:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad7aa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f09 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.527224:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.527227:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.527232:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287ad7aa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f09 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.527245:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:0.0:1788845035.527308:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641622912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5471us (5754us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.527326:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1154 00000100:00100000:3.0:1788845035.530720:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.530865:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641623040 00000100:00100000:1.0:1788845035.530876:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641623040 00000100:00100000:1.0:1788845035.530983:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1155 00000100:00100000:1.0:1788845035.530992:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641623040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.531012:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.531032:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f10 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.531061:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.531070:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.531102:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.531118:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 14 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.531131:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660296960:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845035.531209:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.534680:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.534794:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660296960:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845035.534841:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.534861:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.534864:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.534869:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.534932:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:0.0:1788845035.534989:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641623040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3998us (4278us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845035.534998:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1155 00000100:00100000:3.0:1788845035.535131:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641623168 00000100:00100000:3.0:1788845035.535139:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641623168 00010000:00010000:3.0:1788845035.535168:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 15 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.535209:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 406 00000100:00100000:3.0:1788845035.535220:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641623168:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845035.535233:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845035.535272:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 15 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.535289:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 15 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.535311:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e400/0xb2187301287ad79c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8efb expref: 15 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.535335:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.535340:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.535358:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.535367:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.535401:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.535414:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660297088:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845035.535467:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.541502:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660297088:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845035.541532:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845035.541591:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641623168:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6373us (6920us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.541604:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 406 00000100:00100000:1.0:1788845035.545436:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.545541:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641623296 00000100:00100000:0.0:1788845035.545550:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641623296 00002000:00100000:0.0:1788845035.545600:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d4000 x1875739641623296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.545612:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d4000 with x1875739641623296 00000100:00100000:0.0:1788845035.545651:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 777 00000100:00100000:0.0:1788845035.545661:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641623296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845035.545963:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641623296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 302us (536us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845035.545971:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 777 00002000:00100000:0.0:1788845035.545978:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d4000 x1875739641623296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.545986:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d4000 with x1875739641623296 00000100:00100000:1.0:1788845035.549005:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.549484:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641623424 00000100:00100000:0.0:1788845035.549494:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641623424 00002000:00100000:0.0:1788845035.549541:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d7b80 x1875739641623424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.549552:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d7b80 with x1875739641623424 00000100:00100000:0.0:1788845035.549588:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 778 00000100:00100000:0.0:1788845035.549598:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641623424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845035.549859:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641623424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (862us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845035.549867:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 778 00002000:00100000:0.0:1788845035.549875:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d7b80 x1875739641623424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/4536 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.550164:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d7b80 with x1875739641623424 00000100:00100000:3.0:1788845035.553724:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.553772:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.553930:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641623680 00000100:00100000:1.0:1788845035.553940:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641623680 00000100:00100000:1.0:1788845035.554003:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 790 00000100:00100000:1.0:1788845035.554014:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641623680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.554048:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.554067:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad7b8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f1e expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.554121:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660297216:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.554191:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.556150:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641623552 00000100:00100000:0.0:1788845035.556163:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641623552 00000100:00100000:0.0:1788845035.556220:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1156 00000100:00100000:0.0:1788845035.556230:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641623552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.556250:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.556269:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad7bf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f17 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.556302:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.556455:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660297344:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.556530:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.556757:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660297216:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.556807:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad7b8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f1e expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.556831:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad7b8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f1e expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.556845:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845035.556968:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641623680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2955us (3194us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.556976:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 790 00000100:00100000:0.0:1788845035.560740:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660297344:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.560768:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.561004:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad7bf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f17 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.561027:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.561046:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.561053:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad7bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f17 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.561069:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f600, rc 0) 00000100:00100000:0.0:1788845035.561169:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641623552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4939us (7454us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.561178:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1156 00000100:00100000:2.0:1788845035.565130:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.565202:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.565280:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641623936 00000100:00100000:1.0:1788845035.565290:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641623936 00000100:00100000:1.0:1788845035.565358:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 791 00000100:00100000:1.0:1788845035.565370:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641623936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.565392:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.565414:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad7c6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f2c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.565502:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660297472:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.565592:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.568212:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660297472:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.568252:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad7c6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f2c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.568287:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad7c6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f2c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.568300:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845035.568511:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641623936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3141us (3307us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.568521:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 791 00000100:00100000:0.0:1788845035.568619:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641623808 00000100:00100000:0.0:1788845035.568627:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641623808 00000100:00100000:0.0:1788845035.568672:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1157 00000100:00100000:0.0:1788845035.568692:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641623808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.568711:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.568728:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287ad7cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f25 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.568762:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.568819:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660297600:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.568872:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.571948:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660297600:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.571976:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.571994:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287ad7cd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f25 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.572015:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.572019:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.572024:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287ad7cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f25 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.572040:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:2.0:1788845035.572098:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641623808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3406us (7322us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.572107:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1157 00000100:00100000:2.0:1788845035.576398:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.576602:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641624064 00000100:00100000:3.0:1788845035.576613:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641624064 00000100:00100000:3.0:1788845035.576688:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1158 00000100:00100000:3.0:1788845035.576698:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641624064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.576720:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.576741:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad7d4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f33 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.576780:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.576832:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660297728:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.576960:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.580212:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.580655:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641624192 00000100:00100000:3.0:1788845035.580666:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641624192 00000100:00100000:3.0:1788845035.580733:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 792 00000100:00100000:3.0:1788845035.580744:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641624192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.580762:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.580791:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ad7db lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f3a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.580866:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660297856:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.580976:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.584306:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660297856:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.584362:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ad7db lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f3a expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.584389:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287ad7db lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f3a expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.584419:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:0.0:1788845035.584483:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641624192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3740us (4280us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.584493:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 792 00000100:00100000:3.0:1788845035.586447:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660297728:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845035.586473:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:3.0:1788845035.586492:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad7d4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f33 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.586511:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.586514:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845035.586520:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad7d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f33 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.586534:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e200, rc 0) 00000100:00100000:3.0:1788845035.586599:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641624064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9902us (10210us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845035.586608:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1158 00000100:00100000:1.0:1788845035.589588:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.589631:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.589712:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641624320 00000100:00100000:0.0:1788845035.589722:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641624320 00000100:00100000:0.0:1788845035.589788:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1159 00000100:00100000:0.0:1788845035.589809:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641624320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.589833:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.589856:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad7e2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f41 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.589923:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.589965:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660297984:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.590042:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.590240:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641624448 00000100:00100000:2.0:1788845035.590245:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641624448 00000100:00100000:2.0:1788845035.590275:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 793 00000100:00100000:2.0:1788845035.590283:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641624448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.590292:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.590302:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000016e3697b/0xb2187301287ad7e9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f48 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.590334:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660298112:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.590381:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.593978:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660297984:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.594005:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.594024:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad7e2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f41 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.594043:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.594046:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.594067:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287ad7e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f41 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.594082:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f600, rc 0) 00000100:00100000:0.0:1788845035.594182:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641624320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4384us (4548us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.594191:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1159 00000100:00100000:2.0:1788845035.594296:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660298112:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845035.594329:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000016e3697b/0xb2187301287ad7e9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f48 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.594352:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000016e3697b/0xb2187301287ad7e9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f48 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.594363:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba200, rc 0) 00000100:00100000:2.0:1788845035.594386:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641624448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4104us (4808us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.594392:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 793 00000100:00100000:3.0:1788845035.598563:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.598669:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.598754:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641624704 00000100:00100000:1.0:1788845035.598767:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641624704 00000100:00100000:1.0:1788845035.598839:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 794 00000100:00100000:1.0:1788845035.598850:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641624704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.598871:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.598919:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad7f0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f56 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.598989:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660298240:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.599072:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.601579:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660298240:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.601619:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad7f0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f56 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.601642:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ad7f0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f56 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.601655:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:1.0:1788845035.601725:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641624704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2876us (3055us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.601734:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 794 00000100:00100000:0.0:1788845035.603482:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641624576 00000100:00100000:0.0:1788845035.603493:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641624576 00000100:00100000:1.0:1788845035.603604:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1160 00000100:00100000:1.0:1788845035.603614:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641624576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.603634:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.603653:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad7f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f4f expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.603699:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.603734:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660298368:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.603833:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.605878:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660298368:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845035.605927:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845035.605944:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad7f7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f4f expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.605963:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.605966:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845035.605972:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ad7f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f4f expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.605986:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845035.606042:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641624576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2429us (7490us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.606050:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1160 00000100:00100000:3.0:1788845035.608554:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.608613:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.608696:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641624960 00000100:00100000:0.0:1788845035.608705:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641624960 00000100:00100000:0.0:1788845035.608764:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 795 00000100:00100000:0.0:1788845035.608775:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641624960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.608796:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.608821:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007ff92ced/0xb2187301287ad7fe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f64 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.608902:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660298496:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.608979:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.609594:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641624832 00000100:00100000:1.0:1788845035.609601:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641624832 00000100:00100000:1.0:1788845035.609642:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1161 00000100:00100000:1.0:1788845035.609650:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641624832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.609662:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.609676:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad805 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f5d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.609705:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.609731:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660298624:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.609784:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.612161:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660298496:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.612198:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007ff92ced/0xb2187301287ad7fe lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f64 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.612221:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ff92ced/0xb2187301287ad7fe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f64 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.612252:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e800, rc 0) 00000100:00100000:0.0:1788845035.612316:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641624960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3543us (3771us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.612331:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 795 00000100:00100000:1.0:1788845035.615281:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660298624:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845035.615312:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:1.0:1788845035.615345:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad805 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f5d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.615367:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.615370:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845035.615376:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad805 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f5d expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.615391:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845035.615453:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641624832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5803us (6839us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.615461:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1161 00000100:00100000:1.0:1788845035.618429:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.618529:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641625216 00000100:00100000:3.0:1788845035.618539:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641625216 00000100:00100000:3.0:1788845035.618597:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1162 00000100:00100000:3.0:1788845035.618607:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641625216:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845035.618658:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845035.618696:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ad709 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e6f expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.618714:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.618719:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845035.619165:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.619182:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.619191:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.619226:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.619244:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 10 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.619569:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660298752:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845035.619708:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.624450:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.624640:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660298752:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845035.624666:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.624780:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641625344 00000100:00100000:2.0:1788845035.624787:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641625344 00010000:00010000:2.0:1788845035.624831:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 11 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.624974:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 407 00000100:00100000:2.0:1788845035.624983:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641625344:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845035.624998:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845035.625005:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 11 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.625022:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 11 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.625045:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287ad733 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e92 expref: 11 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.625062:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.625067:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.625076:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.625084:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.625101:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845035.625157:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641625344:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 175us (716us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.625165:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 407 00010000:00010000:2.0:1788845035.626242:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.626300:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845035.626483:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967722 00000004:00010000:2.0:1788845035.626532:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:2.0:1788845035.626536:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.626539:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.626541:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:2.0:1788845035.626544:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000004:00010000:2.0:1788845035.626550:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005c327737 reply state 00000000de310276 transno 4294967723 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.626558:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845035.626812:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641625216:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8195us (8381us total) trans 4294967723 rc 0/0 00000100:00100000:2.0:1788845035.626823:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1162 00000020:00080000:2.0:1788845035.626828:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788845035.634270:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.634288:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.634295:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.634301:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191e200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845035.634306:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.634323:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.634343:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.634346:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845035.634350:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.634355:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e200/0xb2187301287ad80c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.634535:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.634633:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641625472 00000100:00100000:0.0:1788845035.634643:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641625472 00000100:00100000:0.0:1788845035.634705:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1163 00000100:00100000:0.0:1788845035.634715:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641625472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.634737:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.634755:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f6b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.634785:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.634795:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.634836:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.634854:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 9 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.634868:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660298880:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845035.634992:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.637943:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.638029:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641625600 00000100:00100000:3.0:1788845035.638038:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641625600 00002000:00100000:3.0:1788845035.638096:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d4e00 x1875739641625600/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.638108:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d4e00 with x1875739641625600 00010000:00010000:3.0:1788845035.638116:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.638134:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 10 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845035.638143:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a0359d4e00 00000100:00100000:3.0:1788845035.638179:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 27 00000100:00100000:3.0:1788845035.638189:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641625600:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845035.638533:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660298880:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845035.638561:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.638577:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.638580:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.638584:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.638594:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f800, rc 0) 00000100:00100000:0.0:1788845035.638626:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641625472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3913us (4095us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845035.638633:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1163 00000004:00010000:3.0:1788845035.642146:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 582966 00000004:00010000:3.0:1788845035.642156:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:3.0:1788845035.642159:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:3.0:1788845035.642161:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:3.0:1788845035.642164:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:3.0:1788845035.642237:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641625600:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4044us (4298us total) trans 4294967724 rc 0/0 00000100:00100000:3.0:1788845035.642248:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 27 00002000:00100000:3.0:1788845035.642261:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d4e00 x1875739641625600/t4294967724(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/448 e 0 to 0 dl 1788845046 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845035.642270:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d4e00 with x1875739641625600 00010000:00010000:3.0:1788845035.642278:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 12 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.642312:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 12 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845035.642320:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a0359d4e00 00000100:00100000:1.0:1788845035.645767:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.646182:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641625728 00000100:00100000:2.0:1788845035.646192:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641625728 00010000:00010000:2.0:1788845035.646231:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 12 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.646409:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 408 00000100:00100000:2.0:1788845035.646420:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641625728:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845035.646437:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845035.646465:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 12 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.646483:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 12 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.646506:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad7b1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f10 expref: 12 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.646522:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.646529:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.646538:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.646548:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.646589:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.646603:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660299008:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845035.646697:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.651612:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.651869:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660299008:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845035.651913:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845035.651972:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641625728:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5553us (6214us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.651984:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 408 00000100:00100000:3.0:1788845035.652159:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641625856 00000100:00100000:3.0:1788845035.652168:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641625856 00000100:00100000:3.0:1788845035.652223:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 796 00000100:00100000:3.0:1788845035.652233:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641625856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.652250:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.652277:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f72 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.652310:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.652329:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 8) 00010000:00010000:3.0:1788845035.652334:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:3.0:1788845035.652339:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.652369:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.652387:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 5 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.652421:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660299136:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845035.652475:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.658751:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.658989:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641625984 00000100:00100000:2.0:1788845035.658998:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641625984 00010000:00010000:2.0:1788845035.659037:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 6 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.659072:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 409 00000100:00100000:2.0:1788845035.659082:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641625984:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845035.659100:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845035.659124:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 6 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.659143:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 6 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.659172:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:2.0:1788845035.659177:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:2.0:1788845035.659185:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.659195:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.659232:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.659248:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660299264:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845035.659433:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad6fb lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8e68 expref: 6 pid: 6685 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.659449:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660299136:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845035.659475:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.659486:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e200, rc 0) 00000100:00100000:2.0:1788845035.660727:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.661686:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641625856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9436us (10064us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845035.661697:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 796 00000100:00100000:1.0:1788845035.664591:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.665120:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641626112 00000100:00100000:0.0:1788845035.665131:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641626112 00002000:00010000:0.0:1788845035.665174:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1421665->18446744073709551615) 00002000:00100000:0.0:1788845035.665179:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359d7100 with x1875739641626112 ext(1421665->18446744073709551615) 00002000:00010000:0.0:1788845035.665191:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359d7100. 00000100:00100000:0.0:1788845035.665224:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 408 00000100:00100000:0.0:1788845035.665234:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641626112:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845035.665702:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1953551 -> 1421665 00002000:00010000:0.0:1788845035.665707:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00002000:00010000:0.0:1788845035.665710:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00002000:00010000:0.0:1788845035.665712:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00002000:00010000:0.0:1788845035.665729:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:0.0:1788845035.665792:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641626112:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 558us (1210us total) trans 4294967439 rc 0/0 00000100:00100000:0.0:1788845035.665810:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 408 00002000:00010000:0.0:1788845035.665815:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1421665->18446744073709551615) 00002000:00100000:0.0:1788845035.665818:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359d7100 with x1875739641626112 ext(1421665->18446744073709551615) 00002000:00010000:0.0:1788845035.665823:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359d7100. 00000100:00100000:3.0:1788845035.668656:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.668880:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641626240 00000100:00100000:1.0:1788845035.668917:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641626240 00000100:00100000:1.0:1788845035.668970:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1164 00000100:00100000:1.0:1788845035.668981:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641626240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.669000:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.669034:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad821 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f79 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.669123:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad828 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.669138:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad828 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.669151:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad828 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845035.669177:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1421665 00000004:00200000:1.0:1788845035.669181:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845035.669199:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad828 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.669225:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ad821 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f79 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.669241:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad828 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f79 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.669249:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.669251:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845035.669255:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845035.669309:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641626240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 329us (663us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.669315:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1164 00000100:00100000:1.0:1788845035.672691:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.672772:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641626368 00000100:00100000:0.0:1788845035.672781:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641626368 00000100:00100000:0.0:1788845035.672846:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1165 00000100:00100000:0.0:1788845035.672856:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641626368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.672904:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.672924:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287ad82f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f80 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.673007:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.673021:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.673031:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845035.673057:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1421665 00000004:00200000:0.0:1788845035.673061:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845035.673079:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.673106:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287ad82f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.673132:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.673142:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.673145:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845035.673148:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:0.0:1788845035.673205:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641626368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 350us (534us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845035.673212:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1165 00000100:00100000:2.0:1788845035.675758:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660299264:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845035.675789:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845035.675984:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641625984:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 16901us (17241us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845035.675995:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 409 00000100:00100000:0.0:1788845035.678652:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.678788:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.678870:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641626624 00000100:00100000:3.0:1788845035.678909:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641626624 00000100:00100000:3.0:1788845035.678961:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 797 00000100:00100000:3.0:1788845035.678972:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641626624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.678990:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.679009:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad83d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f8e expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.679068:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660299392:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.679159:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.681230:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660299392:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845035.681265:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad83d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f8e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.681288:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad83d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f8e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.681300:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:3.0:1788845035.681440:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641626624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2470us (2651us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845035.681449:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 797 00000100:00100000:1.0:1788845035.681537:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641626496 00000100:00100000:1.0:1788845035.681543:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641626496 00000100:00100000:1.0:1788845035.681583:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1166 00000100:00100000:1.0:1788845035.681590:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641626496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.681606:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.681622:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ad844 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f87 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.681652:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.681824:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbb480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660299520:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.681872:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.685213:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbb480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660299520:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.685240:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.685382:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ad844 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f87 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.685406:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.685409:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.685414:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ad844 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f87 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.685429:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:0.0:1788845035.685546:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641626496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3954us (6902us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.685555:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1166 00000100:00100000:0.0:1788845035.688425:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.688522:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.688594:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641626752 00000100:00100000:2.0:1788845035.688605:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641626752 00000100:00100000:2.0:1788845035.688664:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1167 00000100:00100000:2.0:1788845035.688739:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641626752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.688761:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.688782:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb000/0xb2187301287ad84b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f95 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.688825:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.688868:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660299648:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.689044:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.689144:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641626880 00000100:00100000:1.0:1788845035.689150:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641626880 00000100:00100000:1.0:1788845035.689194:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 798 00000100:00100000:1.0:1788845035.689202:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641626880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.689211:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.689223:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad852 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8f9c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.689271:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660299776:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.689306:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.691827:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660299776:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.691864:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad852 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f9c expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.691930:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ad852 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f9c expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.691943:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:0.0:1788845035.692066:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660299648:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.692082:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:0.0:1788845035.692094:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb000/0xb2187301287ad84b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f95 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.692108:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.692111:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.692115:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbb000/0xb2187301287ad84b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f95 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.692127:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbb000, rc 0) 00000100:00100000:1.0:1788845035.692323:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641626880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3113us (3898us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.692331:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 798 00000100:00100000:0.0:1788845035.692448:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641626752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3775us (3925us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.692455:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1167 00000100:00100000:2.0:1788845035.699536:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.699581:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.699645:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641627136 00000100:00100000:0.0:1788845035.699655:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641627136 00000100:00100000:0.0:1788845035.699720:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 799 00000100:00100000:0.0:1788845035.699731:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641627136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.699751:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.699771:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad859 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8faa expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.699865:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660299904:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.699960:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.701466:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641627008 00000100:00100000:3.0:1788845035.701477:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641627008 00000100:00100000:3.0:1788845035.701534:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1168 00000100:00100000:3.0:1788845035.701543:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641627008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.701562:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.701581:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287ad860 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fa3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.701614:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.701648:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660300032:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.701726:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.706931:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660300032:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845035.706957:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:3.0:1788845035.706977:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287ad860 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fa3 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.706998:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.707000:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845035.707006:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287ad860 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fa3 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.707021:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f400, rc 0) 00000100:00100000:3.0:1788845035.707078:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641627008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5536us (7495us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845035.707086:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1168 00000100:00100000:0.0:1788845035.708399:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660299904:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.708427:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad859 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8faa expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.708445:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287ad859 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8faa expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.708457:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:0.0:1788845035.708500:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641627136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8771us (8974us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.708507:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 799 00000100:00100000:1.0:1788845035.711391:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.711433:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.711499:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641627392 00000100:00100000:0.0:1788845035.711508:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641627392 00000100:00100000:0.0:1788845035.711564:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 800 00000100:00100000:0.0:1788845035.711574:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641627392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.711594:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.711612:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287ad867 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fb8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.711667:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660300160:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.711757:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.711996:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641627264 00000100:00100000:2.0:1788845035.712001:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641627264 00000100:00100000:2.0:1788845035.712040:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1169 00000100:00100000:2.0:1788845035.712048:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641627264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.712059:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.712072:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ad86e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fb1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.712100:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.712123:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660300288:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.712168:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.715153:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660300160:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.715193:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287ad867 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fb8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.715218:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287ad867 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fb8 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.715231:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:0.0:1788845035.715284:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641627392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3711us (3901us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.715293:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 800 00000100:00100000:2.0:1788845035.718181:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660300288:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.718208:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.718226:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ad86e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fb1 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.718245:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.718248:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.718253:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cba000/0xb2187301287ad86e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fb1 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.718289:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cba000, rc 0) 00000100:00100000:2.0:1788845035.718352:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641627264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6304us (6917us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.718360:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1169 00000100:00100000:3.0:1788845035.727011:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.727059:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.727160:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641627520 00000100:00100000:0.0:1788845035.727171:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641627520 00002000:00100000:0.0:1788845035.727219:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d0380 x1875739641627520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.727230:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d0380 with x1875739641627520 00000100:00100000:0.0:1788845035.727275:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 779 00000100:00100000:0.0:1788845035.727284:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641627520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.727914:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641627648 00000100:00100000:1.0:1788845035.727920:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641627648 00002000:00100000:1.0:1788845035.727950:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1425407) req@ffff94a035328a80 x1875739641627648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.727961:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035328a80 with x1875739641627648 ext(1048576->1425407) 00002000:00010000:1.0:1788845035.727965:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035328a80 00000100:00100000:1.0:1788845035.727992:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 409 00000100:00100000:1.0:1788845035.728000:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641627648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.731088:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660300800:0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731144:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845035.731177:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788845035.731183:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660300416:0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731206:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845035.731226:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660300928:0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731270:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845035.731298:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660300544:0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731327:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845035.731342:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660301056:0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731358:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845035.731371:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788845035.731375:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660300672:0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731390:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845035.731415:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660300416 00000100:00100000:1.0:1788845035.731421:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660300416 00000100:00100000:1.0:1788845035.731457:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1788845035.731464:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660300416:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731506:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660300416:12345-0@lo:400:kworker.0 Request processed in 42us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.731512:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1788845035.731539:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660300928 00000100:00100000:1.0:1788845035.731542:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660300928 00000100:00100000:1.0:1788845035.731569:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660300544 00000100:00100000:1.0:1788845035.731571:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660300544 00000100:00100000:1.0:1788845035.731579:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660300672 00000100:00100000:1.0:1788845035.731581:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660300672 00000100:00100000:1.0:1788845035.731593:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1788845035.731597:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660300928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731622:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660300928:12345-0@lo:400:kworker.0 Request processed in 25us (351us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.731628:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1788845035.731636:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1788845035.731647:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660300544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.731946:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660300544:12345-0@lo:400:kworker.0 Request processed in 298us (617us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.731954:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1788845035.731969:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:1.0:1788845035.731976:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660300672:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.732007:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660300672:12345-0@lo:400:kworker.0 Request processed in 32us (615us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.732012:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00080000:1.0:1788845035.732032:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845035.732061:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660300672:0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.732072:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660300544:0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.732101:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660300800 00000100:00100000:1.0:1788845035.732104:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660300800 00000100:00100000:1.0:1788845035.732176:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660301056 00000100:00100000:1.0:1788845035.732180:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660301056 00000100:00080000:2.0:1788845035.732413:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788845035.732416:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845035.732543:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1788845035.732550:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660300800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.732586:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660300800:12345-0@lo:400:kworker.0 Request processed in 37us (1441us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.732592:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1788845035.732603:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:1.0:1788845035.732607:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660301056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845035.732643:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660301056:12345-0@lo:400:kworker.0 Request processed in 35us (1284us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.732661:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:2.0:1788845035.732775:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660301056:0@lo:400:kworker.0 00000100:00100000:0.0:1788845035.735189:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660300416:0@lo:400:kworker.0 00000100:00100000:0.0:1788845035.735212:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660300928:0@lo:400:kworker.0 00000100:00100000:0.0:1788845035.735225:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660300800:0@lo:400:kworker.0 00000100:00100000:0.0:1788845035.744113:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641627520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 16819us (17102us total) trans 0 rc 585728/585728 00000100:00100000:0.0:1788845035.744146:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 779 00002000:00100000:0.0:1788845035.744161:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d0380 x1875739641627520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/440 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 585728/585728 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.744171:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d0380 with x1875739641627520 00000100:00100000:1.0:1788845035.747119:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641627648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 19109us (20048us total) trans 0 rc 376832/376832 00000100:00100000:1.0:1788845035.747133:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 409 00002000:00100000:1.0:1788845035.747147:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1425407) req@ffff94a035328a80 x1875739641627648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:141/0 lens 488/440 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 376832/376832 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.747158:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035328a80 with x1875739641627648 ext(1048576->1425407) 00002000:00010000:1.0:1788845035.747165:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035328a80 00000100:00100000:0.0:1788845035.752947:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.752987:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.753052:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641627904 00000100:00100000:3.0:1788845035.753062:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641627904 00000100:00100000:3.0:1788845035.753173:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 801 00000100:00100000:3.0:1788845035.753184:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641627904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.753202:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.753220:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287ad875 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fc6 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.753275:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660301184:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.753359:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.753859:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641627776 00000100:00100000:1.0:1788845035.753865:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641627776 00000100:00100000:1.0:1788845035.753930:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1170 00000100:00100000:1.0:1788845035.753939:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641627776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.753951:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.753965:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad87c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fbf expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.753994:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.754020:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660301312:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.754064:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.756401:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660301312:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.756430:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 582966 00010000:00010000:2.0:1788845035.756450:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad87c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fbf expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.756471:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.756474:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.756479:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ad87c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fbf expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.756495:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:2.0:1788845035.756553:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641627776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2615us (3617us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.756561:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1170 00000100:00100000:1.0:1788845035.756956:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660301184:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.756977:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287ad875 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fc6 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.757016:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006614a3bc/0xb2187301287ad875 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fc6 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.757026:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f400, rc 0) 00000100:00100000:1.0:1788845035.757058:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641627904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3877us (4071us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.757066:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 801 00000100:00100000:3.0:1788845035.760244:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.760356:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641628032 00000100:00100000:2.0:1788845035.760366:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641628032 00000100:00100000:2.0:1788845035.760423:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1171 00000100:00100000:2.0:1788845035.760433:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641628032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.760455:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.760477:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fcd expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.760507:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.760516:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.760553:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.760572:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 16 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.760585:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660301440:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845035.760662:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.763074:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660301440:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845035.763109:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.763129:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.763132:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.763137:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.763153:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008cbbc00, rc 0) 00000100:00100000:2.0:1788845035.763211:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641628032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2779us (2978us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.763220:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1171 00000100:00100000:2.0:1788845035.766531:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.766681:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641628160 00000100:00100000:3.0:1788845035.766690:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641628160 00010000:00010000:3.0:1788845035.766726:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 17 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.766761:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 410 00000100:00100000:3.0:1788845035.766770:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641628160:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845035.766782:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845035.766824:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 17 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.766842:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 17 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.766863:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287ad813 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f6b expref: 17 pid: 5841 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.766915:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.766922:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.766934:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.766943:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.766981:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.767013:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660301568:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845035.767086:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.770107:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660301568:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845035.770135:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845035.770192:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641628160:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3422us (3669us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.770203:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 410 00000100:00100000:0.0:1788845035.776046:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.776150:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641628288 00000100:00100000:1.0:1788845035.776160:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641628288 00002000:00100000:1.0:1788845035.776212:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48380 x1875739641628288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.776224:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48380 with x1875739641628288 00000100:00100000:1.0:1788845035.776261:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 780 00000100:00100000:1.0:1788845035.776271:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641628288:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845035.803111:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641628288:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 26832us (27067us total) trans 0 rc 585728/585728 00000100:00100000:2.0:1788845035.803126:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 780 00002000:00100000:2.0:1788845035.803141:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48380 x1875739641628288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/440 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 585728/585728 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845035.803151:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48380 with x1875739641628288 00000100:00100000:3.0:1788845035.806881:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.807014:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641628416 00000100:00100000:0.0:1788845035.807024:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641628416 00000100:00100000:0.0:1788845035.807071:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 802 00000100:00100000:0.0:1788845035.807081:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641628416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.807102:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.807120:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ad88a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fd4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.807185:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660301696:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.807270:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.809773:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660301696:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.809823:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ad88a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fd4 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.809849:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ad88a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fd4 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.809933:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:0.0:1788845035.809992:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641628416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2912us (3120us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.810002:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 802 00000100:00100000:3.0:1788845035.812939:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.813092:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641628544 00000100:00100000:0.0:1788845035.813102:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641628544 00000100:00100000:0.0:1788845035.813152:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 803 00000100:00100000:0.0:1788845035.813160:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641628544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.813178:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.813192:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad891 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fdb expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.813241:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660301824:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.813755:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.815579:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660301824:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.815621:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad891 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fdb expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.815648:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad891 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fdb expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.815663:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845035.815732:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641628544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2572us (2847us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.815758:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 803 00000100:00100000:3.0:1788845035.818772:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.818872:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641628672 00000100:00100000:1.0:1788845035.818919:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641628672 00002000:00100000:1.0:1788845035.818971:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532a680 x1875739641628672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.818982:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532a680 with x1875739641628672 00000100:00100000:1.0:1788845035.819013:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 781 00000100:00100000:1.0:1788845035.819024:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641628672:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845035.819273:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641628672:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 249us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.819291:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 781 00002000:00100000:1.0:1788845035.819298:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532a680 x1875739641628672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/440 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845035.819306:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532a680 with x1875739641628672 00000100:00100000:3.0:1788845035.822347:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.822444:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641628800 00000100:00100000:1.0:1788845035.822452:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641628800 00000100:00100000:1.0:1788845035.822505:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 804 00000100:00100000:1.0:1788845035.822514:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641628800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.822533:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.822550:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ad898 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fe2 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.822604:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660301952:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.822684:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.824974:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660301952:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.825014:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ad898 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fe2 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.825041:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ad898 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fe2 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.825053:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845035.825113:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641628800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2599us (2776us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.825141:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 804 00000100:00100000:3.0:1788845035.828050:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.828153:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641628928 00000100:00100000:0.0:1788845035.828164:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641628928 00002000:00100000:0.0:1788845035.828216:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532b100 x1875739641628928/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/0 e 0 to 0 dl 1788845046 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.828229:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532b100 with x1875739641628928 00000100:00100000:0.0:1788845035.828273:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 782 00000100:00100000:0.0:1788845035.828283:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641628928:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845035.831648:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 582966 -> 589824 00000004:00010000:0.0:1788845035.831658:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:0.0:1788845035.831660:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:0.0:1788845035.831663:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:0.0:1788845035.831665:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 776 -> 776 00000100:00100000:0.0:1788845035.833079:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641628928:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4784us (5025us total) trans 4294967725 rc 0/0 00000100:00100000:0.0:1788845035.833097:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 782 00002000:00100000:0.0:1788845035.833114:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532b100 x1875739641628928/t4294967725(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:141/0 lens 488/448 e 0 to 0 dl 1788845046 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845035.833125:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532b100 with x1875739641628928 00000100:00100000:2.0:1788845035.835878:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.835969:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641629056 00000100:00100000:0.0:1788845035.835980:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641629056 00000100:00100000:0.0:1788845035.836040:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1172 00000100:00100000:0.0:1788845035.836051:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641629056:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845035.864952:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967725 is committed 00000001:00080000:3.0:1788845035.864975:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967723 is committed 00000100:00100000:2.0:1788845035.866420:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641629056:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 30362us (30544us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.866435:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1172 00000100:00100000:2.0:1788845035.869927:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.870137:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641629184 00000100:00100000:1.0:1788845035.870147:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641629184 00000100:00100000:1.0:1788845035.870202:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 805 00000100:00100000:1.0:1788845035.870212:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641629184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.870230:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.870249:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad89f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8fe9 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.870309:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660302080:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.870520:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.873559:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660302080:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845035.873605:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad89f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fe9 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.873630:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad89f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fe9 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.873643:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:3.0:1788845035.873824:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641629184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3611us (3906us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845035.873834:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 805 00000100:00100000:2.0:1788845035.877127:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.877225:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641629312 00000100:00100000:0.0:1788845035.877251:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641629312 00000100:00100000:0.0:1788845035.877442:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 806 00000100:00100000:0.0:1788845035.877453:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641629312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.877471:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.877489:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad8a6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ff0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.877547:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660302208:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.877680:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.880117:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660302208:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.880155:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad8a6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ff0 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.880180:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad8a6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ff0 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.880193:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845035.880292:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641629312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2840us (3174us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.880301:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 806 00000100:00100000:2.0:1788845035.883608:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845035.883697:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641629440 00000100:00100000:3.0:1788845035.883707:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641629440 00000100:00100000:3.0:1788845035.883766:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1173 00000100:00100000:3.0:1788845035.883777:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641629440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845035.883825:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845035.883846:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287ad8ad lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ff7 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.884098:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.884142:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660302336:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845035.884228:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788845035.886240:0:5841:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 776 -> 840 00000100:00100000:3.0:1788845035.886282:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660302336:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845035.886305:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 589824 00010000:00010000:3.0:1788845035.886333:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287ad8ad lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ff7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.886352:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845035.886355:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845035.886360:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fa00/0xb2187301287ad8ad lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ff7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845035.886374:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:3.0:1788845035.886431:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641629440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2655us (2833us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845035.886440:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1173 00000100:00100000:1.0:1788845035.889146:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.889240:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641629568 00000100:00100000:2.0:1788845035.889250:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641629568 00000100:00100000:0.0:1788845035.889448:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1174 00000100:00100000:0.0:1788845035.889460:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641629568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.889481:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.889501:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad8b4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a8ffe expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.889562:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.889601:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660302464:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.889692:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.894457:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660302464:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.894484:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 589824 00010000:00010000:0.0:1788845035.894502:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad8b4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ffe expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.894521:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.894523:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.894528:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad8b4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8ffe expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.894542:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845035.894605:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641629568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5147us (5467us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.894612:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1174 00000100:00100000:3.0:1788845035.897436:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.897514:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641629696 00000100:00100000:2.0:1788845035.897522:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641629696 00000100:00100000:2.0:1788845035.897568:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1175 00000100:00100000:2.0:1788845035.897575:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641629696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.897593:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.897609:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ad8bb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9005 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.897639:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.897681:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660302592:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.897775:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.900060:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660302592:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.900085:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 589824 00010000:00010000:2.0:1788845035.900102:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ad8bb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9005 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.900119:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.900121:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.900125:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ad8bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9005 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.900136:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845035.900213:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641629696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2638us (2785us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.900219:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1175 00000100:00100000:1.0:1788845035.902571:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.902635:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641629824 00000100:00100000:2.0:1788845035.902643:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641629824 00000100:00100000:2.0:1788845035.902758:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1176 00000100:00100000:2.0:1788845035.902766:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641629824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.902788:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.902816:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ad8c2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a900c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.902854:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.902906:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660302720:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.902974:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.904930:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660302720:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845035.904961:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 589824 00010000:00010000:1.0:1788845035.904979:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ad8c2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a900c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.905001:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.905004:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845035.905009:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ad8c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a900c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.905025:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032800, rc 0) 00000100:00100000:1.0:1788845035.905079:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641629824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2312us (2513us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.905102:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1176 00000100:00100000:3.0:1788845035.907863:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.908127:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641629952 00000100:00100000:0.0:1788845035.908137:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641629952 00000100:00100000:0.0:1788845035.908249:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1177 00000100:00100000:0.0:1788845035.908260:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641629952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.908281:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.908303:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad8c9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9013 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.908371:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.908411:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660302848:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.908494:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.910664:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660302848:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.910700:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 589824 00010000:00010000:0.0:1788845035.910718:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad8c9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9013 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.910737:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.910740:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.910745:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad8c9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9013 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.910760:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845035.910824:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641629952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2565us (2970us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.910832:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1177 00000100:00100000:3.0:1788845035.913565:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.913679:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641630080 00000100:00100000:0.0:1788845035.913690:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641630080 00000100:00100000:0.0:1788845035.913749:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1178 00000100:00100000:0.0:1788845035.913759:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641630080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.913780:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.913808:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8d0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a901a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.913854:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.913925:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660302976:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.913996:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.916494:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660302976:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.916515:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 589824 00010000:00010000:0.0:1788845035.916531:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8d0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a901a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.916546:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.916548:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.916551:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a901a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.916563:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:0.0:1788845035.916612:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641630080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2856us (3058us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.916619:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1178 00000100:00100000:3.0:1788845035.919295:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.919394:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641630208 00000100:00100000:0.0:1788845035.919403:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641630208 00000100:00100000:0.0:1788845035.919449:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 807 00000100:00100000:0.0:1788845035.919456:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641630208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.919471:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.919485:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ad8d7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9021 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.919543:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660303104:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.919611:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.921993:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660303104:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.922035:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ad8d7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9021 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.922061:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ad8d7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9021 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.922074:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:0.0:1788845035.922141:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641630208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2683us (2852us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.922150:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 807 00000100:00100000:1.0:1788845035.926221:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.926310:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641630336 00000100:00100000:0.0:1788845035.926329:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641630336 00000100:00100000:0.0:1788845035.926388:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1179 00000100:00100000:0.0:1788845035.926397:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641630336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.926419:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.926441:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8de lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9028 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.926479:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.926541:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660303232:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.926625:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.928442:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660303232:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.928462:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 589824 00010000:00010000:0.0:1788845035.928476:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8de lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9028 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.928491:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.928494:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.928496:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8de lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9028 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.928507:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:0.0:1788845035.928554:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641630336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2159us (2343us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.928560:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1179 00000100:00100000:3.0:1788845035.930838:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845035.930998:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641630464 00000100:00100000:2.0:1788845035.931008:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641630464 00000100:00100000:2.0:1788845035.931069:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1180 00000100:00100000:2.0:1788845035.931080:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641630464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845035.931100:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845035.931121:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ad8e5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a902f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.931157:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.931197:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660303360:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845035.931289:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.933234:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660303360:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845035.933274:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 589824 00010000:00010000:2.0:1788845035.933295:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ad8e5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a902f expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.933326:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.933330:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845035.933335:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ad8e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a902f expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.933352:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845035.933412:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641630464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2334us (2585us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845035.933420:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1180 00000100:00100000:3.0:1788845035.935970:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.936102:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641630592 00000100:00100000:1.0:1788845035.936110:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641630592 00000100:00100000:1.0:1788845035.936153:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 808 00000100:00100000:1.0:1788845035.936160:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641630592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.936176:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.936191:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad8ec lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9036 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.936239:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660303488:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.936319:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.938328:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660303488:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.938358:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad8ec lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9036 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.938376:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287ad8ec lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9036 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.938386:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845035.938433:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641630592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2273us (2473us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.938439:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 808 00000100:00100000:3.0:1788845035.940560:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.940619:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641630720 00000100:00100000:1.0:1788845035.940639:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641630720 00000100:00100000:1.0:1788845035.940687:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 809 00000100:00100000:1.0:1788845035.940694:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641630720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.940708:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.940723:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad8f3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a903d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.940765:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660303616:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845035.940846:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.946366:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660303616:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.946411:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad8f3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a903d expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.946436:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad8f3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a903d expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.946449:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845035.946510:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641630720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5814us (5954us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.946520:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 809 00000100:00100000:3.0:1788845035.949368:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.949518:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641630976 00000100:00100000:0.0:1788845035.949528:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641630976 00000100:00100000:0.0:1788845035.949587:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1181 00000100:00100000:0.0:1788845035.949598:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641630976:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845035.949653:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845035.949693:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287ad828 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f79 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.949713:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.949732:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845035.950067:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.950082:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.950091:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.950126:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.950145:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 10 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.950159:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660303744:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845035.950249:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.955039:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.955143:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641631104 00000100:00100000:1.0:1788845035.955154:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641631104 00010000:00010000:1.0:1788845035.955196:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 11 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845035.955245:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 411 00000100:00100000:1.0:1788845035.955280:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d4b850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641631104:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845035.955299:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845035.955306:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 11 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.955332:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 11 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.955359:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.955365:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.955376:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.955385:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.955404:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845035.955457:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d4b850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641631104:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 180us (428us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.955465:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 411 00010000:00010000:0.0:1788845035.956002:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287ad836 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f80 expref: 10 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.956021:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660303744:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845035.956039:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.956048:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.956054:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845035.956181:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967726 00000004:00010000:0.0:1788845035.956231:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 589824 -> 589824 00000004:00010000:0.0:1788845035.956235:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:0.0:1788845035.956238:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:0.0:1788845035.956240:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:0.0:1788845035.956242:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 840 -> 840 00000004:00010000:0.0:1788845035.956249:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005492b202 reply state 000000004b166d67 transno 4294967727 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.956257:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845035.956344:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641630976:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6747us (6985us total) trans 4294967727 rc 0/0 00000100:00100000:0.0:1788845035.956352:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1181 00000020:00080000:0.0:1788845035.956356:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845035.959163:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.959179:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.959187:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.959212:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02030200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845035.959217:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.959225:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.959242:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.959245:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845035.959250:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.959269:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ad8fa lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845035.960729:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.960789:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641631232 00000100:00100000:0.0:1788845035.960806:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641631232 00000100:00100000:0.0:1788845035.960868:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1182 00000100:00100000:0.0:1788845035.960879:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641631232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.960941:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.960962:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9044 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.960990:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.961000:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.961046:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.961065:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 9 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.961079:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660303872:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845035.961152:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.966114:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.966233:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660303872:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845035.966278:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.966299:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.966302:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.966307:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.966328:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:0.0:1788845035.966390:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641631232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5512us (5666us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845035.966399:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1182 00000100:00100000:2.0:1788845035.966606:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641631360 00000100:00100000:2.0:1788845035.966614:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641631360 00010000:00010000:2.0:1788845035.966645:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.966693:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 412 00000100:00100000:2.0:1788845035.966702:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641631360:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845035.966715:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845035.966740:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.966756:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.966784:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008cbbc00/0xb2187301287ad883 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8fcd expref: 10 pid: 11036 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.966809:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845035.966814:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.966824:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.966834:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845035.966870:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845035.966917:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660304000:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845035.966970:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0034a4180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845035.969782:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660304000:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845035.969818:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845035.969874:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641631360:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3171us (3770us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845035.969912:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 412 00000100:00100000:3.0:1788845035.972212:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.972309:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641631488 00000100:00100000:1.0:1788845035.972330:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641631488 00002000:00010000:1.0:1788845035.972373:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845035.972379:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455500 with x1875739641631488 00000100:00100000:1.0:1788845035.972417:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 783 00000100:00100000:1.0:1788845035.972427:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641631488:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845035.972758:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 589824 -> 403464 00000004:00010000:1.0:1788845035.972762:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00000004:00010000:1.0:1788845035.972765:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:1.0:1788845035.972767:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00000004:00010000:1.0:1788845035.972770:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 840 -> 840 00000100:00100000:1.0:1788845035.972855:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641631488:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 429us (650us total) trans 4294967728 rc 0/0 00000100:00100000:1.0:1788845035.973224:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 783 00002000:00010000:1.0:1788845035.973229:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845035.973232:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03455500 with x1875739641631488 00000100:00100000:3.0:1788845035.974849:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.974992:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641631616 00000100:00100000:0.0:1788845035.975001:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641631616 00002000:00010000:0.0:1788845035.975034:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845035.975039:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03454e00 with x1875739641631616 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845035.975072:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03454e00. 00000100:00100000:0.0:1788845035.975104:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 410 00000100:00100000:0.0:1788845035.975113:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641631616:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845035.975475:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1421665 -> 0 00002000:00010000:0.0:1788845035.975479:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845035 -> 1788845035 00002000:00010000:0.0:1788845035.975482:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00002000:00010000:0.0:1788845035.975484:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845035 -> 1788845035 00002000:00010000:0.0:1788845035.975486:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:0.0:1788845035.975539:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641631616:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 426us (696us total) trans 4294967440 rc 0/0 00000100:00100000:0.0:1788845035.975546:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 410 00002000:00010000:0.0:1788845035.975549:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845035.975552:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03454e00 with x1875739641631616 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845035.975556:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03454e00. 00000100:00100000:3.0:1788845035.978962:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845035.979285:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641631744 00000100:00100000:1.0:1788845035.979296:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641631744 00000100:00100000:1.0:1788845035.979368:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1183 00000100:00100000:1.0:1788845035.979378:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641631744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845035.979402:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845035.979425:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ad908 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a904b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.979516:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ad90f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.979531:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ad90f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.979541:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ad90f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845035.979570:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 403464 00000004:00200000:1.0:1788845035.979574:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845035.979593:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ad90f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.979620:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ad908 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a904b expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.979638:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ad90f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a904b expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.979982:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845035.979989:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845035.979993:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845035.980066:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641631744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 689us (1114us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845035.980074:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1183 00000100:00100000:3.0:1788845035.982296:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.982388:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641631872 00000100:00100000:0.0:1788845035.982396:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641631872 00000100:00100000:0.0:1788845035.982440:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1184 00000100:00100000:0.0:1788845035.982447:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641631872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.982465:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.982480:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad916 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9052 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.982510:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.982541:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660304128:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.982618:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.983350:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.983919:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641632000 00000100:00100000:0.0:1788845035.983927:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641632000 00000100:00100000:0.0:1788845035.983974:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 810 00000100:00100000:0.0:1788845035.983983:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641632000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.983999:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.984014:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ad91d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9059 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.984061:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660304256:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.984125:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845035.986030:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660304128:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.986057:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 403464 00010000:00010000:0.0:1788845035.986075:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad916 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9052 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.986094:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.986097:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.986102:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad916 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9052 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.986118:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845035.986174:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641631872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3724us (3884us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.986183:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1184 00000100:00100000:0.0:1788845035.987951:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660304256:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845035.988001:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ad91d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9059 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.988024:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ad91d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9059 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.988036:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:0.0:1788845035.988078:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641632000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4096us (4730us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.988086:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 810 00000100:00100000:3.0:1788845035.991552:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.991618:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641632128 00000100:00100000:0.0:1788845035.991627:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641632128 00000100:00100000:0.0:1788845035.991684:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1185 00000100:00100000:0.0:1788845035.991695:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641632128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.991718:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.991738:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad924 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9060 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.991774:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.991826:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660304384:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.991934:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845035.994404:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845035.994496:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641632256 00000100:00100000:0.0:1788845035.994505:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641632256 00000100:00100000:0.0:1788845035.994560:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 811 00000100:00100000:0.0:1788845035.994585:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641632256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845035.994604:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845035.994621:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ad92b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9067 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845035.994686:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660304512:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845035.994765:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845035.996920:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660304512:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845035.996958:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ad92b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9067 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.996983:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ad92b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9067 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845035.996997:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:1.0:1788845035.997054:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641632256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2470us (2660us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845035.997062:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 811 00000100:00100000:0.0:1788845035.997547:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660304384:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845035.997560:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 403464 00010000:00010000:0.0:1788845035.997572:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad924 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9060 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.997585:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845035.997588:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845035.997592:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad924 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9060 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845035.997603:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:0.0:1788845035.997638:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641632128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5945us (6096us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845035.997645:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1185 00000100:00100000:1.0:1788845036.001813:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.001922:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641632384 00000100:00100000:0.0:1788845036.001933:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641632384 00000100:00100000:0.0:1788845036.001987:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1186 00000100:00100000:0.0:1788845036.001997:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641632384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.002023:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.002059:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad932 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a906e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.002154:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.002170:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.002182:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845036.002207:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 403464 00000004:00200000:0.0:1788845036.002212:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845036.002228:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.002254:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad932 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.002272:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.002281:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.002283:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845036.002287:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:0.0:1788845036.002357:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641632384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 362us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.002364:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1186 00000100:00100000:3.0:1788845036.011125:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.011228:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641632512 00000100:00100000:1.0:1788845036.011238:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641632512 00000100:00100000:1.0:1788845036.011297:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 812 00000100:00100000:1.0:1788845036.011306:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641632512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.011329:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.011346:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9075 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.011390:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.011399:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(4 8) 00010000:00010000:1.0:1788845036.011404:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 8) 00010000:00010000:1.0:1788845036.011408:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1421312->1486847] (req 1421312->1486847) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.011439:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.011456:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 7 pid: 7983 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.011470:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660304640:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845036.011544:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.013433:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660304640:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.013467:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1421312->1486847] (req 1421312->1486847) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.013487:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845036.013545:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641632512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2239us (2428us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.013570:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 812 00000100:00100000:3.0:1788845036.016069:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.016223:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641632640 00000100:00100000:1.0:1788845036.016232:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641632640 00010000:00010000:1.0:1788845036.016286:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 8 pid: 7983 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.016391:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 413 00000100:00100000:1.0:1788845036.016401:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641632640:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845036.016417:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845036.016440:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 8 pid: 7983 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.016459:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 8 pid: 7983 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.016484:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ad81a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a8f72 expref: 8 pid: 7983 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.016501:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 8) 00010000:00010000:1.0:1788845036.016505:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 8) 00010000:00010000:1.0:1788845036.016513:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.016523:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.016580:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.016594:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660304768:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845036.017169:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.019390:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660304768:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845036.019417:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845036.019472:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641632640:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3072us (3412us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.019484:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 413 00000100:00100000:3.0:1788845036.020635:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.020724:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641632768 00000100:00100000:1.0:1788845036.020732:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641632768 00000100:00100000:1.0:1788845036.020800:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1187 00000100:00100000:1.0:1788845036.020810:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641632768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.020836:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.020855:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad947 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a907c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.020929:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.020967:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660304896:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.021047:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.023640:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660304896:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.023691:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 403464 00010000:00010000:1.0:1788845036.023711:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad947 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a907c expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.023734:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.023738:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.023744:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ad947 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a907c expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.023759:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845036.023827:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641632768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3017us (3200us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.023836:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1187 00000100:00100000:3.0:1788845036.026267:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.026428:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641632896 00000100:00100000:0.0:1788845036.026450:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641632896 00000100:00100000:0.0:1788845036.026504:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1188 00000100:00100000:0.0:1788845036.026514:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641632896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.026535:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.026640:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad94e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9083 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.026699:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.026746:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660305024:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.026858:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.030615:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660305024:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.030642:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 403464 00010000:00010000:0.0:1788845036.030662:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad94e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9083 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.030681:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.030684:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.030689:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ad94e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9083 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.030705:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845036.030768:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641632896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4254us (4519us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.030787:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1188 00000100:00100000:3.0:1788845036.034405:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.034639:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641633024 00000100:00100000:1.0:1788845036.034649:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641633024 00000100:00100000:1.0:1788845036.034749:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 813 00000100:00100000:1.0:1788845036.034760:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641633024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.034779:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.034805:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad955 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a908a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.034863:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660305152:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.034978:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00002000:00010000:1.0:1788845036.036983:0:7983:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1485187 00002000:00010000:1.0:1788845036.036992:0:7983:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845035 -> 1788845036 00002000:00010000:1.0:1788845036.036994:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845035 -> 1788845036 00000100:00100000:1.0:1788845036.037019:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660305152:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.037067:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad955 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a908a expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.037091:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ad955 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a908a expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.037105:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845036.037194:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641633024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2435us (2798us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.037203:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 813 00000100:00100000:2.0:1788845036.042034:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.042189:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641633152 00000100:00100000:3.0:1788845036.042199:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641633152 00000100:00100000:3.0:1788845036.042262:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 814 00000100:00100000:3.0:1788845036.042273:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641633152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.042295:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.042615:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad95c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9091 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.042698:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ceea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660305280:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.042787:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.046037:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ceea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660305280:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.046069:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad95c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9091 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.046088:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad95c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9091 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.046097:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:3.0:1788845036.046171:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641633152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3901us (4149us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.046179:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 814 00000100:00100000:2.0:1788845036.048438:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.048526:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641633280 00000100:00100000:0.0:1788845036.048550:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641633280 00000100:00100000:0.0:1788845036.048607:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 815 00000100:00100000:0.0:1788845036.048617:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641633280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.048637:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.048655:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad963 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9098 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.048723:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660305408:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.048841:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.051207:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660305408:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845036.051243:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad963 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9098 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.051270:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ad963 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9098 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.051284:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845036.051333:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641633280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2717us (2901us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.051342:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 815 00000100:00100000:2.0:1788845036.054533:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.054629:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641633408 00000100:00100000:0.0:1788845036.054639:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641633408 00002000:00100000:0.0:1788845036.054680:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687800 x1875739641633408/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.054691:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687800 with x1875739641633408 00000100:00100000:0.0:1788845036.054746:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 784 00000100:00100000:0.0:1788845036.054756:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641633408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.055271:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641633408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 515us (747us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.055279:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 784 00002000:00100000:0.0:1788845036.055286:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687800 x1875739641633408/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.055294:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687800 with x1875739641633408 00000100:00100000:2.0:1788845036.059686:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.059851:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641633536 00000100:00100000:0.0:1788845036.059862:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641633536 00002000:00100000:0.0:1788845036.059930:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684380 x1875739641633536/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.059943:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684380 with x1875739641633536 00000100:00100000:0.0:1788845036.059987:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 785 00000100:00100000:0.0:1788845036.059997:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641633536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.060204:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641633536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 207us (527us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.060212:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 785 00002000:00100000:0.0:1788845036.060219:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684380 x1875739641633536/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.060226:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684380 with x1875739641633536 00000100:00100000:1.0:1788845036.063023:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.063116:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641633664 00000100:00100000:3.0:1788845036.063126:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641633664 00002000:00100000:3.0:1788845036.063174:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db8380 x1875739641633664/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.063186:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db8380 with x1875739641633664 00000100:00100000:3.0:1788845036.063219:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 786 00000100:00100000:3.0:1788845036.063230:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641633664:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.063566:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641633664:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 335us (550us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.063575:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 786 00002000:00100000:3.0:1788845036.063597:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db8380 x1875739641633664/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.063605:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db8380 with x1875739641633664 00000100:00100000:2.0:1788845036.066655:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.066836:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641633792 00000100:00100000:1.0:1788845036.066847:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641633792 00002000:00100000:1.0:1788845036.066966:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685c00 x1875739641633792/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.066978:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685c00 with x1875739641633792 00000100:00100000:1.0:1788845036.067009:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 787 00000100:00100000:1.0:1788845036.067020:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641633792:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.067235:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641633792:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 216us (589us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.067244:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 787 00002000:00100000:1.0:1788845036.067251:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685c00 x1875739641633792/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.067259:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685c00 with x1875739641633792 00000100:00100000:0.0:1788845036.069859:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.069913:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641633920 00000100:00100000:1.0:1788845036.069924:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641633920 00002000:00100000:1.0:1788845036.069970:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741880 x1875739641633920/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.069981:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741880 with x1875739641633920 00000100:00100000:1.0:1788845036.070018:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 788 00000100:00100000:1.0:1788845036.070027:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641633920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.070236:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641633920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 209us (385us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.070244:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 788 00002000:00100000:1.0:1788845036.070251:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741880 x1875739641633920/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.070259:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741880 with x1875739641633920 00000100:00100000:0.0:1788845036.073559:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.073681:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641634048 00000100:00100000:1.0:1788845036.073692:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641634048 00002000:00100000:1.0:1788845036.073740:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741f80 x1875739641634048/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.073751:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741f80 with x1875739641634048 00000100:00100000:1.0:1788845036.073801:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 789 00000100:00100000:1.0:1788845036.073811:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641634048:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.074067:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641634048:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (516us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.074075:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 789 00002000:00100000:1.0:1788845036.074083:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741f80 x1875739641634048/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.074111:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741f80 with x1875739641634048 00000100:00100000:0.0:1788845036.077173:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.077344:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641634176 00000100:00100000:1.0:1788845036.077353:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641634176 00002000:00100000:1.0:1788845036.077392:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035742d80 x1875739641634176/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.077401:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742d80 with x1875739641634176 00000100:00100000:1.0:1788845036.077457:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 790 00000100:00100000:1.0:1788845036.077464:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641634176:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.077660:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641634176:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 194us (498us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.077677:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 790 00002000:00100000:1.0:1788845036.077684:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035742d80 x1875739641634176/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.077692:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742d80 with x1875739641634176 00000100:00100000:2.0:1788845036.080591:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.080683:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641634304 00000100:00100000:0.0:1788845036.080692:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641634304 00002000:00100000:0.0:1788845036.080737:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686300 x1875739641634304/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.080749:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686300 with x1875739641634304 00000100:00100000:0.0:1788845036.080783:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 791 00000100:00100000:0.0:1788845036.080801:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641634304:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.081081:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641634304:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 289us (499us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.081089:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 791 00002000:00100000:0.0:1788845036.081096:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686300 x1875739641634304/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.081104:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686300 with x1875739641634304 00000100:00100000:0.0:1788845036.084282:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.084357:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641634432 00000100:00100000:3.0:1788845036.084367:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641634432 00002000:00100000:3.0:1788845036.084412:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035740000 x1875739641634432/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.084420:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035740000 with x1875739641634432 00000100:00100000:3.0:1788845036.084448:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 792 00000100:00100000:3.0:1788845036.084455:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641634432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.084688:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641634432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (414us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.084694:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 792 00002000:00100000:3.0:1788845036.084699:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035740000 x1875739641634432/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.084704:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035740000 with x1875739641634432 00000100:00100000:2.0:1788845036.087663:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.087917:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641634560 00000100:00100000:3.0:1788845036.087927:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641634560 00002000:00100000:3.0:1788845036.087974:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685500 x1875739641634560/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.087985:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685500 with x1875739641634560 00000100:00100000:3.0:1788845036.088026:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 793 00000100:00100000:3.0:1788845036.088036:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641634560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.088349:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641634560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 312us (694us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.088373:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 793 00002000:00100000:3.0:1788845036.088381:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685500 x1875739641634560/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.088389:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685500 with x1875739641634560 00000100:00100000:2.0:1788845036.092133:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.092228:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641634688 00000100:00100000:3.0:1788845036.092240:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641634688 00002000:00100000:3.0:1788845036.092438:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684000 x1875739641634688/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.092451:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684000 with x1875739641634688 00000100:00100000:3.0:1788845036.092528:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 794 00000100:00100000:3.0:1788845036.092539:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641634688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.092824:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641634688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 285us (699us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.092832:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 794 00002000:00100000:3.0:1788845036.092839:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684000 x1875739641634688/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.092848:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684000 with x1875739641634688 00000100:00100000:0.0:1788845036.096552:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.096644:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641634816 00000100:00100000:3.0:1788845036.096655:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641634816 00002000:00100000:3.0:1788845036.096764:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741180 x1875739641634816/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.096775:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741180 with x1875739641634816 00000100:00100000:3.0:1788845036.096866:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 795 00000100:00100000:3.0:1788845036.096877:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641634816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.097221:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641634816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 344us (677us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.097229:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 795 00002000:00100000:3.0:1788845036.097236:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035741180 x1875739641634816/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.097245:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035741180 with x1875739641634816 00000100:00100000:2.0:1788845036.100450:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.100540:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641634944 00000100:00100000:0.0:1788845036.100549:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641634944 00002000:00100000:0.0:1788845036.100597:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684e00 x1875739641634944/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.100607:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684e00 with x1875739641634944 00000100:00100000:0.0:1788845036.100636:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 796 00000100:00100000:0.0:1788845036.100643:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641634944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.100942:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641634944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 298us (501us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.100948:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 796 00002000:00100000:0.0:1788845036.100961:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684e00 x1875739641634944/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.100969:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684e00 with x1875739641634944 00000100:00100000:2.0:1788845036.104122:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.104213:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641635072 00000100:00100000:1.0:1788845036.104223:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641635072 00002000:00100000:1.0:1788845036.104266:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686d80 x1875739641635072/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.104277:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686d80 with x1875739641635072 00000100:00100000:1.0:1788845036.104315:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 797 00000100:00100000:1.0:1788845036.104324:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641635072:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.104563:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641635072:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 239us (448us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.104586:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 797 00002000:00100000:1.0:1788845036.104593:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686d80 x1875739641635072/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.104601:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686d80 with x1875739641635072 00000100:00100000:1.0:1788845036.108162:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.108247:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641635200 00000100:00100000:0.0:1788845036.108257:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641635200 00002000:00100000:0.0:1788845036.108306:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db9880 x1875739641635200/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.108327:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db9880 with x1875739641635200 00000100:00100000:0.0:1788845036.108363:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 798 00000100:00100000:0.0:1788845036.108374:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641635200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.108651:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641635200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (498us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.108659:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 798 00002000:00100000:0.0:1788845036.108678:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01db9880 x1875739641635200/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.108686:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01db9880 with x1875739641635200 00000100:00100000:2.0:1788845036.111627:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.111916:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641635328 00000100:00100000:1.0:1788845036.111925:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641635328 00002000:00100000:1.0:1788845036.111969:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684a80 x1875739641635328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.111980:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684a80 with x1875739641635328 00000100:00100000:1.0:1788845036.112019:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 799 00000100:00100000:1.0:1788845036.112027:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641635328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.112247:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641635328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 220us (627us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.112252:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 799 00002000:00100000:1.0:1788845036.112257:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684a80 x1875739641635328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.112263:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684a80 with x1875739641635328 00010000:00010000:2.0:1788845036.114593:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845036.115347:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.115482:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641635456 00000100:00100000:0.0:1788845036.115492:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641635456 00002000:00100000:0.0:1788845036.115537:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687480 x1875739641635456/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.115549:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687480 with x1875739641635456 00000100:00100000:0.0:1788845036.115588:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 800 00000100:00100000:0.0:1788845036.115598:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641635456:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845036.119128:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 403464 -> 403464 00000004:00010000:1.0:1788845036.119136:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845035 -> 1788845036 00000004:00010000:1.0:1788845036.119138:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00000004:00010000:1.0:1788845036.119140:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845035 -> 1788845036 00000004:00010000:1.0:1788845036.119142:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 840 -> 840 00000100:00100000:1.0:1788845036.119200:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2368:x1875739641635456:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3601us (3858us total) trans 4294967729 rc 0/0 00000100:00100000:1.0:1788845036.119215:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 800 00002000:00100000:1.0:1788845036.119224:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687480 x1875739641635456/t4294967729(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/448 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.119231:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687480 with x1875739641635456 00000100:00100000:2.0:1788845036.121810:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.122162:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641635584 00000100:00100000:0.0:1788845036.122170:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641635584 00000100:00100000:1.0:1788845036.122488:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1189 00000100:00100000:1.0:1788845036.122496:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641635584:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845036.138409:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967727 is committed 00000001:00080000:3.0:1788845036.138430:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967729 is committed 00000100:00100000:0.0:1788845036.138566:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641635584:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 16065us (16770us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.138579:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1189 00000100:00100000:2.0:1788845036.141032:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.141133:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641635712 00000100:00100000:1.0:1788845036.141142:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641635712 00000100:00100000:1.0:1788845036.141193:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 816 00000100:00100000:1.0:1788845036.141203:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641635712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.141223:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.141240:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad96a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a909f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.141293:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660305536:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.141442:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.143503:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660305536:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.143541:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad96a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a909f expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.143565:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad96a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a909f expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.143599:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845036.143697:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641635712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2495us (2675us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.143706:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 816 00000100:00100000:2.0:1788845036.152175:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.154751:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641635840 00000100:00100000:1.0:1788845036.154770:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641635840 00000100:00100000:1.0:1788845036.154845:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 817 00000100:00100000:1.0:1788845036.154855:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641635840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.154874:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.155069:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad971 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90a6 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.155135:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660305664:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.155265:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.157270:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660305664:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.157302:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad971 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90a6 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.157325:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ad971 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90a6 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.157337:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845036.157414:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641635840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2561us (5251us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.157433:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 817 00000100:00100000:2.0:1788845036.159810:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.159968:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641635968 00000100:00100000:0.0:1788845036.159978:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641635968 00000100:00100000:0.0:1788845036.160065:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1190 00000100:00100000:0.0:1788845036.160076:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641635968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.160097:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.160117:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ad978 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90ad expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.160151:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.160191:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660305792:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.160278:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000004:00010000:0.0:1788845036.162650:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845035 -> 1788845036 00000100:00100000:0.0:1788845036.162691:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660305792:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.162713:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 403464 00010000:00010000:0.0:1788845036.162731:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ad978 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90ad expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.162751:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.162754:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.162759:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ad978 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90ad expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.162806:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:0.0:1788845036.162863:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641635968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2789us (3074us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.162872:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1190 00000100:00100000:2.0:1788845036.165718:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.165828:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641636096 00000100:00100000:1.0:1788845036.165843:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641636096 00000100:00100000:1.0:1788845036.165927:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1191 00000100:00100000:1.0:1788845036.165938:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641636096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.165960:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.165983:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90b4 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.166014:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.166025:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.166065:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.166084:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 14 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.166097:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660305920:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845036.166160:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.168619:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660305920:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.168656:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.168705:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.168709:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.168714:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.168724:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845036.168786:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641636096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2848us (3081us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.168804:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1191 00000100:00100000:2.0:1788845036.170331:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.170415:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641636224 00000100:00100000:0.0:1788845036.170424:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641636224 00010000:00010000:0.0:1788845036.170459:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 15 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.170499:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 414 00000100:00100000:0.0:1788845036.170509:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641636224:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845036.170523:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845036.170551:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 15 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.170569:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 15 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.170593:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad901 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9044 expref: 15 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.170610:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.170617:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.170644:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.170654:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.170703:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.170717:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660306048:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845036.170801:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.172719:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660306048:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845036.172742:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845036.172798:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641636224:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2282us (2465us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.172808:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 414 00000100:00100000:3.0:1788845036.177323:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.177371:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.177484:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641636352 00000100:00100000:1.0:1788845036.177493:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641636352 00002000:00100000:1.0:1788845036.177542:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01ceea00 x1875739641636352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.177550:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01ceea00 with x1875739641636352 00000100:00100000:1.0:1788845036.177575:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 801 00000100:00100000:1.0:1788845036.177582:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641636352:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.178085:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641636480 00000100:00100000:0.0:1788845036.178091:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641636480 00002000:00100000:0.0:1788845036.178121:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1421311) req@ffff949f01cef480 x1875739641636480/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.178131:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cef480 with x1875739641636480 ext(1048576->1421311) 00002000:00010000:0.0:1788845036.178136:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cef480 00000100:00100000:0.0:1788845036.178165:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 411 00000100:00100000:0.0:1788845036.178173:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641636480:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.181581:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641636352:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 3986us (4268us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.181595:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 801 00002000:00100000:1.0:1788845036.181608:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01ceea00 x1875739641636352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/440 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.181619:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01ceea00 with x1875739641636352 00000100:00100000:2.0:1788845036.186273:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.186355:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641636608 00000100:00100000:3.0:1788845036.186365:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641636608 00002000:00100000:3.0:1788845036.186411:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be4000 x1875739641636608/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.186422:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be4000 with x1875739641636608 00000100:00100000:3.0:1788845036.186462:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 802 00000100:00100000:3.0:1788845036.186472:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641636608:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845036.189717:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 403464 -> 991232 00000004:00010000:1.0:1788845036.189724:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.189734:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845035 00000004:00010000:1.0:1788845036.189735:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.189737:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 840 -> 584 00000100:00100000:1.0:1788845036.189815:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641636608:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3339us (3546us total) trans 4294967730 rc 0/0 00000100:00100000:1.0:1788845036.189825:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 802 00002000:00100000:1.0:1788845036.189837:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be4000 x1875739641636608/t4294967730(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/448 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.189846:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be4000 with x1875739641636608 00000100:00100000:0.0:1788845036.190729:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641636480:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12554us (13355us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.190737:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 411 00002000:00100000:0.0:1788845036.190743:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1421311) req@ffff949f01cef480 x1875739641636480/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/440 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.190749:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cef480 with x1875739641636480 ext(1048576->1421311) 00002000:00010000:0.0:1788845036.190753:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cef480 00000100:00100000:3.0:1788845036.192391:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.192858:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641636736 00000100:00100000:1.0:1788845036.192870:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641636736 00000100:00100000:1.0:1788845036.193030:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1192 00000100:00100000:1.0:1788845036.193040:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641636736:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845036.222630:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967730 is committed 00000100:00100000:0.0:1788845036.222863:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641636736:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 29818us (30474us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.222877:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1192 00000100:00100000:1.0:1788845036.226602:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.226681:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641636864 00000100:00100000:0.0:1788845036.226689:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641636864 00000100:00100000:0.0:1788845036.226736:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1193 00000100:00100000:0.0:1788845036.226743:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641636864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.226761:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.226777:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad986 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90bb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.226829:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.226869:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660306176:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.227171:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.227867:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.228110:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641636992 00000100:00100000:0.0:1788845036.228118:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641636992 00000100:00100000:0.0:1788845036.228187:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 818 00000100:00100000:0.0:1788845036.228196:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641636992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.228210:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.228226:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ad98d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90c2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.228270:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660306304:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.228424:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000004:00010000:0.0:1788845036.229152:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845035 -> 1788845036 00000004:00010000:0.0:1788845036.229158:0:5841:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 584 -> 712 00000100:00100000:0.0:1788845036.229179:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660306176:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.229198:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 991232 00010000:00010000:0.0:1788845036.229215:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad986 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90bb expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.229233:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.229236:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.229240:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ad986 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90bb expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.229255:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:0.0:1788845036.229294:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641636864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2551us (2700us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.229301:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1193 00002000:00010000:0.0:1788845036.231419:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845035 -> 1788845036 00000100:00100000:0.0:1788845036.231444:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660306304:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845036.231491:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ad98d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c2 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.231511:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ad98d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c2 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.231520:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:0.0:1788845036.231596:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641636992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3401us (3733us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.231602:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 818 00000100:00100000:1.0:1788845036.233630:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.233878:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641637120 00000100:00100000:0.0:1788845036.234080:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641637120 00000100:00100000:1.0:1788845036.234201:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 819 00000100:00100000:1.0:1788845036.234209:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641637120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.234225:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.234242:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90c9 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.234281:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.234290:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 8) 00010000:00010000:1.0:1788845036.234415:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:1.0:1788845036.234420:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1159168->1163263] (req 1159168->1163263) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.234449:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.234464:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 5 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.234474:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660306432:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845036.234583:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a012d330c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.239004:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660306432:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.239046:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1159168->1163263] (req 1159168->1163263) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.239067:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:1.0:1788845036.239120:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641637120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4910us (5497us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.239128:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 819 00000100:00100000:2.0:1788845036.241180:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.241264:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641637248 00000100:00100000:0.0:1788845036.241274:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641637248 00002000:00100000:0.0:1788845036.241334:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1421312->1485186) req@ffff94a035be4e00 x1875739641637248/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.241345:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be4e00 with x1875739641637248 ext(1421312->1485186) 00010000:00010000:0.0:1788845036.241354:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 6 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.241374:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 6 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845036.241380:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a035be4e00 00000100:00100000:0.0:1788845036.241406:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 21 00000100:00100000:0.0:1788845036.241413:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641637248:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845036.242167:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845036.242187:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660306560:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845036.242238:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845036.242424:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660306560 00000100:00100000:2.0:1788845036.242430:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660306560 00000100:00100000:2.0:1788845036.242854:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 81 00000100:00100000:2.0:1788845036.242866:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660306560:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845036.243205:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 117506048 pending 1703936 free 3859808256 avail 3854565376 00000100:00100000:2.0:1788845036.243266:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660306560:12345-0@lo:13:osp-pre-1-0.0 Request processed in 402us (1026us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845036.243274:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 81 00000100:00100000:0.0:1788845036.243306:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660306560:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845036.245089:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641637248:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3665us (3907us total) trans 4294967441 rc 0/0 00000100:00100000:0.0:1788845036.245101:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 21 00002000:00100000:0.0:1788845036.245113:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1421312->1485186) req@ffff94a035be4e00 x1875739641637248/t4294967441(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/448 e 0 to 0 dl 1788845047 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.245122:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be4e00 with x1875739641637248 ext(1421312->1485186) 00010000:00010000:0.0:1788845036.245133:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 8 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.245156:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 8 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845036.245165:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a035be4e00 00000100:00100000:3.0:1788845036.248530:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.248783:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641637376 00000100:00100000:2.0:1788845036.248805:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641637376 00010000:00010000:2.0:1788845036.248850:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 8 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.248910:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 415 00000100:00100000:2.0:1788845036.248920:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641637376:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845036.248936:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845036.248960:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 8 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.248977:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 8 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.248999:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ad940 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1421312->1486847) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9075 expref: 8 pid: 6684 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.249015:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:2.0:1788845036.249020:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:2.0:1788845036.249028:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.249039:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.249078:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.249098:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660306688:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845036.249172:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.251647:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.251725:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641637504 00000100:00100000:0.0:1788845036.251736:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641637504 00002000:00100000:0.0:1788845036.251785:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1159168->1486847) req@ffff949f01cedc00 x1875739641637504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 504/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.251806:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cedc00 with x1875739641637504 ext(1159168->1486847) 00002000:00010000:0.0:1788845036.251813:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cedc00 00000100:00100000:0.0:1788845036.251847:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 412 00000100:00100000:2.0:1788845036.251850:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660306688:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845036.251856:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641637504:12345-192.168.201.29@tcp:3:fsx.0 00010000:00010000:2.0:1788845036.251872:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845036.251939:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641637376:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3020us (3417us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845036.251947:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 415 00000100:00100000:0.0:1788845036.256172:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641637504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4305us (4520us total) trans 0 rc 65536/65536 00000100:00100000:0.0:1788845036.256206:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 412 00002000:00100000:0.0:1788845036.256218:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1159168->1486847) req@ffff949f01cedc00 x1875739641637504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 504/440 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.256228:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cedc00 with x1875739641637504 ext(1159168->1486847) 00002000:00010000:0.0:1788845036.256235:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cedc00 00000100:00100000:0.0:1788845036.258749:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.258841:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641637632 00000100:00100000:2.0:1788845036.258851:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641637632 00002000:00100000:2.0:1788845036.258917:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1163264->1167359) req@ffff94a035741500 x1875739641637632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.258927:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035741500 with x1875739641637632 ext(1163264->1167359) 00002000:00010000:2.0:1788845036.258933:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035741500 00000100:00100000:2.0:1788845036.258970:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 413 00000100:00100000:2.0:1788845036.258979:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641637632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845036.259207:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641637632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 228us (466us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845036.259215:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 413 00002000:00100000:2.0:1788845036.259221:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1163264->1167359) req@ffff94a035741500 x1875739641637632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.259229:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035741500 with x1875739641637632 ext(1163264->1167359) 00002000:00010000:2.0:1788845036.259232:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035741500 00000100:00100000:3.0:1788845036.261407:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.261469:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641637760 00000100:00100000:0.0:1788845036.261479:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641637760 00002000:00100000:0.0:1788845036.261523:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1167360->1171455) req@ffff949f01cec000 x1875739641637760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.261535:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cec000 with x1875739641637760 ext(1167360->1171455) 00002000:00010000:0.0:1788845036.261540:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cec000 00000100:00100000:0.0:1788845036.261626:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 414 00000100:00100000:0.0:1788845036.261637:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641637760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.261920:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641637760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 247us (481us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.261928:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 414 00002000:00100000:0.0:1788845036.261934:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1167360->1171455) req@ffff949f01cec000 x1875739641637760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.261942:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cec000 with x1875739641637760 ext(1167360->1171455) 00002000:00010000:0.0:1788845036.261945:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cec000 00000100:00100000:3.0:1788845036.264592:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.264685:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641637888 00000100:00100000:1.0:1788845036.264695:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641637888 00002000:00100000:1.0:1788845036.264743:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1171456->1175551) req@ffff949f01ceed80 x1875739641637888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.264753:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01ceed80 with x1875739641637888 ext(1171456->1175551) 00002000:00010000:1.0:1788845036.264772:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01ceed80 00000100:00100000:1.0:1788845036.264814:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 415 00000100:00100000:1.0:1788845036.264824:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641637888:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.265161:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641637888:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 336us (575us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.265169:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 415 00002000:00100000:1.0:1788845036.265176:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1171456->1175551) req@ffff949f01ceed80 x1875739641637888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.265185:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01ceed80 with x1875739641637888 ext(1171456->1175551) 00002000:00010000:1.0:1788845036.265189:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01ceed80 00000100:00100000:3.0:1788845036.268255:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.268432:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641638016 00000100:00100000:0.0:1788845036.268442:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641638016 00002000:00100000:0.0:1788845036.268489:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1175552->1179647) req@ffff949f01cefb80 x1875739641638016/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.268501:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cefb80 with x1875739641638016 ext(1175552->1179647) 00002000:00010000:0.0:1788845036.268506:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cefb80 00000100:00100000:0.0:1788845036.268586:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 416 00000100:00100000:0.0:1788845036.268597:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.268850:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (603us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.268857:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 416 00002000:00100000:0.0:1788845036.268865:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1175552->1179647) req@ffff949f01cefb80 x1875739641638016/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.268872:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cefb80 with x1875739641638016 ext(1175552->1179647) 00002000:00010000:0.0:1788845036.268875:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cefb80 00000100:00100000:3.0:1788845036.271699:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.271771:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641638144 00000100:00100000:2.0:1788845036.271782:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641638144 00002000:00100000:2.0:1788845036.271840:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1179648->1183743) req@ffff949f01ced500 x1875739641638144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.271852:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01ced500 with x1875739641638144 ext(1179648->1183743) 00002000:00010000:2.0:1788845036.271857:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01ced500 00000100:00100000:0.0:1788845036.271944:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 417 00000100:00100000:0.0:1788845036.271956:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.272208:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (518us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.274650:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.277046:0:6688:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 3 min 3 max 116 00000100:00100000:1.0:1788845036.277046:0:6689:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 3 min 3 max 116 00000100:00100000:1.0:1788845036.277063:0:6689:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_003' 00000100:00100000:2.0:1788845036.277067:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641638272 00000100:00100000:2.0:1788845036.277077:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641638272 00002000:00100000:2.0:1788845036.277129:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1183744->1187839) req@ffff94a03568aa00 x1875739641638272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.277140:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03568aa00 with x1875739641638272 ext(1183744->1187839) 00002000:00010000:2.0:1788845036.277147:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03568aa00 00000100:00100000:0.0:1788845036.277260:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 417 00002000:00100000:0.0:1788845036.277272:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1179648->1183743) req@ffff949f01ced500 x1875739641638144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.277281:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01ced500 with x1875739641638144 ext(1179648->1183743) 00002000:00010000:0.0:1788845036.277285:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01ced500 00000100:00100000:0.0:1788845036.277383:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 418 00000100:00100000:0.0:1788845036.277397:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.277760:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 365us (3118us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.277768:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 418 00002000:00100000:0.0:1788845036.277775:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1183744->1187839) req@ffff94a03568aa00 x1875739641638272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.277783:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03568aa00 with x1875739641638272 ext(1183744->1187839) 00002000:00010000:0.0:1788845036.277817:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03568aa00 00000100:00100000:3.0:1788845036.280922:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.281008:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641638400 00000100:00100000:1.0:1788845036.281018:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641638400 00002000:00100000:1.0:1788845036.281066:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1187840->1191935) req@ffff94a035688700 x1875739641638400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.281077:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035688700 with x1875739641638400 ext(1187840->1191935) 00002000:00010000:1.0:1788845036.281083:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035688700 00000100:00100000:1.0:1788845036.281118:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 419 00000100:00100000:1.0:1788845036.281127:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641638400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.281572:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641638400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 445us (658us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.281580:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 419 00002000:00100000:1.0:1788845036.281588:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1187840->1191935) req@ffff94a035688700 x1875739641638400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.281608:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035688700 with x1875739641638400 ext(1187840->1191935) 00002000:00010000:1.0:1788845036.281612:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035688700 00000100:00100000:3.0:1788845036.285092:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.285346:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641638528 00000100:00100000:0.0:1788845036.285356:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641638528 00002000:00100000:0.0:1788845036.285401:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1191936->1196031) req@ffff94a035689c00 x1875739641638528/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.285412:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035689c00 with x1875739641638528 ext(1191936->1196031) 00002000:00010000:0.0:1788845036.285417:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035689c00 00000100:00100000:0.0:1788845036.285450:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 420 00000100:00100000:0.0:1788845036.285460:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638528:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.285683:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638528:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 224us (602us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.285691:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 420 00002000:00100000:0.0:1788845036.285697:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1191936->1196031) req@ffff94a035689c00 x1875739641638528/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.285705:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035689c00 with x1875739641638528 ext(1191936->1196031) 00002000:00010000:0.0:1788845036.285708:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035689c00 00000100:00100000:0.0:1788845036.289037:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.289131:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641638656 00000100:00100000:1.0:1788845036.289141:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641638656 00002000:00100000:1.0:1788845036.289190:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1196032->1200127) req@ffff94a035741c00 x1875739641638656/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.289202:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035741c00 with x1875739641638656 ext(1196032->1200127) 00002000:00010000:1.0:1788845036.289208:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035741c00 00000100:00100000:1.0:1788845036.289242:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 421 00000100:00100000:1.0:1788845036.289252:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641638656:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.289500:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641638656:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (471us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.289508:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 421 00002000:00100000:1.0:1788845036.289516:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1196032->1200127) req@ffff94a035741c00 x1875739641638656/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.289524:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035741c00 with x1875739641638656 ext(1196032->1200127) 00002000:00010000:1.0:1788845036.289528:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035741c00 00000100:00100000:2.0:1788845036.292510:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.292610:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641638784 00000100:00100000:0.0:1788845036.292620:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641638784 00002000:00100000:0.0:1788845036.292727:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1200128->1204223) req@ffff94a035be6680 x1875739641638784/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.292739:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be6680 with x1875739641638784 ext(1200128->1204223) 00002000:00010000:0.0:1788845036.292745:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be6680 00000100:00100000:0.0:1788845036.292781:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 422 00000100:00100000:0.0:1788845036.292802:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638784:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.293176:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641638784:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 385us (675us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.293184:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 422 00002000:00100000:0.0:1788845036.293191:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1200128->1204223) req@ffff94a035be6680 x1875739641638784/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.293200:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be6680 with x1875739641638784 ext(1200128->1204223) 00002000:00010000:0.0:1788845036.293204:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be6680 00000100:00100000:0.0:1788845036.296559:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.296654:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641638912 00000100:00100000:1.0:1788845036.296672:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641638912 00002000:00100000:1.0:1788845036.296721:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1204224->1208319) req@ffff94a035743100 x1875739641638912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.296731:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035743100 with x1875739641638912 ext(1204224->1208319) 00002000:00010000:1.0:1788845036.296737:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035743100 00000100:00100000:1.0:1788845036.296795:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 423 00000100:00100000:1.0:1788845036.296806:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641638912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.297107:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641638912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 301us (558us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.297115:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 423 00002000:00100000:1.0:1788845036.297122:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1204224->1208319) req@ffff94a035743100 x1875739641638912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.297130:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035743100 with x1875739641638912 ext(1204224->1208319) 00002000:00010000:1.0:1788845036.297133:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035743100 00000100:00100000:3.0:1788845036.300635:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.300924:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641639040 00000100:00100000:1.0:1788845036.300934:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641639040 00000100:00100000:1.0:1788845036.300984:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 820 00000100:00100000:1.0:1788845036.300994:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641639040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.301014:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.301033:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90d0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.301056:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:1.0:1788845036.301060:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:1.0:1788845036.301067:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.301081:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.301107:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845036.301253:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641639040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 259us (624us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.301270:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 820 00000100:00100000:2.0:1788845036.304031:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.304133:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641639296 00000100:00100000:0.0:1788845036.304143:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641639296 00000100:00100000:0.0:1788845036.304206:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1194 00000100:00100000:0.0:1788845036.304216:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641639296:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845036.304281:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845036.304318:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ad90f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a904b expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.305730:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.305735:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845036.306071:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.306090:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.306098:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.306137:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.306161:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 13 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.306176:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660306816:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845036.306271:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.309680:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.309915:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660306816:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845036.309945:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.310008:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641639424 00000100:00100000:3.0:1788845036.310015:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641639424 00010000:00010000:3.0:1788845036.310049:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 14 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.310082:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 416 00000100:00100000:3.0:1788845036.310091:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641639424:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845036.310108:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845036.310114:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 14 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.310130:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 14 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.310152:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ad939 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a906e expref: 14 pid: 5840 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.310169:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.310175:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.310198:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.310206:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.310235:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845036.310465:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.310473:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845036.310588:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967731 00000004:00010000:2.0:1788845036.310640:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991232 -> 991232 00000004:00010000:2.0:1788845036.310644:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:2.0:1788845036.310647:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845035 00000004:00010000:2.0:1788845036.310650:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:2.0:1788845036.310652:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:2.0:1788845036.310659:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001afa2d83 reply state 00000000ee325219 transno 4294967732 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.310679:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845036.310800:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641639296:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6584us (6777us total) trans 4294967732 rc 0/0 00000100:00100000:2.0:1788845036.310808:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1194 00000020:00080000:2.0:1788845036.310813:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845036.312479:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641639424:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2370us (2799us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845036.312492:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 416 00010000:00010000:3.0:1788845036.313351:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313364:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313371:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313377:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02031c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845036.313382:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313390:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313405:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.313409:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845036.313413:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313439:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ad9a2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.313584:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.313764:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641639552 00000100:00100000:3.0:1788845036.313774:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641639552 00000100:00100000:3.0:1788845036.313865:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 821 00000100:00100000:3.0:1788845036.313875:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641639552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.313916:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.313933:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90d7 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313964:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313973:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.313980:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:3.0:1788845036.313984:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:3.0:1788845036.313988:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.314020:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.314038:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 9 pid: 6684 timeout: 525 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.314051:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660306944:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845036.314128:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.316867:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 7 pid: 6685 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.316922:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660307072:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845036.317229:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.317692:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.317767:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641639680 00000100:00100000:3.0:1788845036.317775:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641639680 00010000:00010000:3.0:1788845036.317824:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 10 pid: 6684 timeout: 525 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.317866:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 417 00000100:00100000:3.0:1788845036.317877:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641639680:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845036.317920:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845036.317945:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 10 pid: 6684 timeout: 525 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.317963:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 10 pid: 6684 timeout: 525 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.317990:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845036.318036:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641639680:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 160us (345us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845036.318045:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 417 00010000:00010000:2.0:1788845036.321696:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ad99b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d0 expref: 9 pid: 6684 timeout: 525 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.321721:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660306944:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845036.321737:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660307072:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845036.321770:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 8 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.321799:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fa00, rc 0) 00000100:00100000:2.0:1788845036.321944:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641639552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8070us (8361us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845036.321953:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 821 00000100:00100000:1.0:1788845036.324531:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.324763:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641639808 00000100:00100000:2.0:1788845036.324773:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641639808 00010000:00010000:2.0:1788845036.324823:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 8 pid: 6685 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.324865:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 418 00000100:00100000:2.0:1788845036.324875:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d49050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641639808:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845036.324917:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845036.324941:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 8 pid: 6685 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.324960:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 8 pid: 6685 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.324983:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ad994 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1159168->1163263) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90c9 expref: 8 pid: 6685 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.325001:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:2.0:1788845036.325005:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:2.0:1788845036.325015:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.325025:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.325064:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.325082:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660307200:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845036.325151:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.330583:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.330626:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.330657:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641640064 00000100:00100000:0.0:1788845036.330665:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641640064 00002000:00010000:0.0:1788845036.330714:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845036.330720:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c4bb80 with x1875739641640064 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845036.330734:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c4bb80. 00000100:00100000:0.0:1788845036.330764:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 424 00000100:00100000:0.0:1788845036.330775:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641640064:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845036.331211:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1485187 -> 0 00002000:00010000:0.0:1788845036.331215:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00002000:00010000:0.0:1788845036.331217:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845036 -> 1788845035 00002000:00010000:0.0:1788845036.331219:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00002000:00010000:0.0:1788845036.331221:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:0.0:1788845036.331279:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641640064:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 505us (651us total) trans 4294967442 rc 0/0 00000100:00100000:0.0:1788845036.331287:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 424 00002000:00010000:0.0:1788845036.331291:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845036.331294:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c4bb80 with x1875739641640064 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845036.331298:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c4bb80. 00000100:00100000:2.0:1788845036.331458:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660307200:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845036.331480:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845036.331507:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d49050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641639808:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6633us (6985us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845036.331516:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 418 00000100:00100000:1.0:1788845036.331624:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641639936 00000100:00100000:1.0:1788845036.331629:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641639936 00002000:00010000:1.0:1788845036.331652:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845036.331657:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48e00 with x1875739641639936 00000100:00100000:1.0:1788845036.331687:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 803 00000100:00100000:1.0:1788845036.331694:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641639936:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845036.331953:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 991232 -> 371399 00000004:00010000:1.0:1788845036.331957:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.331960:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845035 -> 1788845036 00000004:00010000:1.0:1788845036.331962:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.331964:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845036.331996:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641639936:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 303us (1423us total) trans 4294967733 rc 0/0 00000100:00100000:1.0:1788845036.332002:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 803 00002000:00010000:1.0:1788845036.332005:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845036.332007:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48e00 with x1875739641639936 00000100:00100000:0.0:1788845036.336738:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.336827:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641640192 00000100:00100000:1.0:1788845036.336837:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641640192 00000100:00100000:1.0:1788845036.337006:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1195 00000100:00100000:1.0:1788845036.337032:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641640192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.337058:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.337082:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad9b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90de expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.337182:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad9b7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.337197:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad9b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.337208:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad9b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845036.337235:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 371399 00000004:00200000:1.0:1788845036.337240:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845036.337271:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad9b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.337299:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad9b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90de expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.337326:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad9b7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90de expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.337335:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.337338:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845036.337342:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845036.337395:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641640192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 365us (731us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.337412:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1195 00000100:00100000:3.0:1788845036.340992:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.341499:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641640320 00000100:00100000:0.0:1788845036.341509:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641640320 00000100:00100000:0.0:1788845036.341573:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1196 00000100:00100000:0.0:1788845036.341585:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641640320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.341606:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.341628:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ad9be lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90e5 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.341727:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.341746:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.341757:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845036.341794:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 371399 00000004:00200000:0.0:1788845036.341799:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845036.341818:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.341843:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ad9be lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.341861:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.341870:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.341873:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845036.341877:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:0.0:1788845036.341951:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641640320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 369us (970us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.341958:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1196 00000100:00100000:3.0:1788845036.344960:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.345003:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.345091:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641640448 00000100:00100000:0.0:1788845036.345102:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641640448 00000100:00100000:0.0:1788845036.345154:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1197 00000100:00100000:0.0:1788845036.345165:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641640448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.345186:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.345209:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ad9cc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90ec expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.345247:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.345288:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660307328:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.345380:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.345397:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641640576 00000100:00100000:0.0:1788845036.345400:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641640576 00000100:00100000:0.0:1788845036.345432:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 822 00000100:00100000:0.0:1788845036.345440:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641640576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.345448:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.345460:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ad9d3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90f3 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.345495:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660307456:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.345521:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.347990:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660307328:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.348018:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:0.0:1788845036.348035:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ad9cc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90ec expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.348056:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.348059:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.348064:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ad9cc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90ec expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.348085:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:0.0:1788845036.348143:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641640448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2980us (3192us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.348151:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1197 00002000:00010000:3.0:1788845036.349446:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845035 -> 1788845036 00000100:00100000:3.0:1788845036.349470:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660307456:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.349497:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ad9d3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90f3 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.349518:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ad9d3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90f3 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.349530:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031a00, rc 0) 00000100:00100000:3.0:1788845036.349580:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641640576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4140us (4576us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.349588:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 822 00000100:00100000:3.0:1788845036.352144:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.352183:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.352210:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641640704 00000100:00100000:3.0:1788845036.352218:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641640704 00000100:00100000:3.0:1788845036.352262:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1198 00000100:00100000:3.0:1788845036.352272:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641640704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.352294:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.352329:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad9da lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a90fa expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.352366:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.352407:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660307584:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.352478:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.352929:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641640832 00000100:00100000:1.0:1788845036.352935:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641640832 00000100:00100000:1.0:1788845036.352971:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 823 00000100:00100000:1.0:1788845036.352979:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641640832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.352989:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.353001:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad9e1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9101 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.353038:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660307712:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.353072:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.356654:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660307712:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.356705:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad9e1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9101 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.356729:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ad9e1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9101 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.356741:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845036.356812:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641640832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3832us (4677us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.356820:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 823 00000100:00100000:3.0:1788845036.357708:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660307584:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845036.357729:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:3.0:1788845036.357744:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad9da lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90fa expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.357759:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.357762:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845036.357767:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad9da lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90fa expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.357779:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845036.357837:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641640704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5566us (5652us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.357845:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1198 00000100:00100000:0.0:1788845036.363620:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.363682:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.363830:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641641088 00000100:00100000:2.0:1788845036.363841:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641641088 00000100:00100000:2.0:1788845036.363935:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 824 00000100:00100000:2.0:1788845036.363946:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641641088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.363965:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.363985:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ad9e8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a910f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.364046:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660307840:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.364119:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.364240:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641640960 00000100:00100000:1.0:1788845036.364245:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641640960 00000100:00100000:1.0:1788845036.364297:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1199 00000100:00100000:1.0:1788845036.364304:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641640960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.364315:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.364329:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad9ef lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9108 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.364362:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.364387:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660307968:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.364420:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.368075:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660307968:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.368106:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:1.0:1788845036.368127:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad9ef lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9108 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.368148:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.368151:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.368156:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ad9ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9108 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.368172:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845036.368229:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641640960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3925us (4621us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.368238:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1199 00000100:00100000:2.0:1788845036.368829:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660307840:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845036.368856:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ad9e8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a910f expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.368874:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ad9e8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a910f expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.368945:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845036.368985:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641641088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5041us (5302us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845036.368993:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 824 00000100:00100000:0.0:1788845036.373228:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.373451:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641641344 00000100:00100000:3.0:1788845036.373461:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641641344 00000100:00100000:3.0:1788845036.373517:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 825 00000100:00100000:3.0:1788845036.373527:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641641344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.373547:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.373566:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad9f6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a911d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.373631:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660308096:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.374209:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.374448:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.377035:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641641216 00000100:00100000:3.0:1788845036.377051:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641641216 00000100:00100000:3.0:1788845036.377121:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1200 00000100:00100000:3.0:1788845036.377131:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641641216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.377159:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.377185:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad9fd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9116 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.377230:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.377282:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660308224:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.377519:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660308096:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.377550:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad9f6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a911d expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.377588:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd628f2c/0xb2187301287ad9f6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a911d expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.377603:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f000, rc 0) 00000100:00100000:1.0:1788845036.377635:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641641344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4109us (4418us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.377643:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 825 00000100:00100000:1.0:1788845036.378870:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.385157:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660308224:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845036.385186:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:3.0:1788845036.385208:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad9fd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9116 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.385229:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.385232:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845036.385238:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f800/0xb2187301287ad9fd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9116 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.385401:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845036.385476:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641641216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8345us (11027us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.385485:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1200 00000100:00100000:2.0:1788845036.388233:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.388329:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641641472 00000100:00100000:1.0:1788845036.388338:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641641472 00000100:00100000:1.0:1788845036.388413:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 804 00000100:00100000:1.0:1788845036.388423:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641641472:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845036.388682:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 371399 -> 371399 00000004:00010000:1.0:1788845036.388687:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.388690:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.388692:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.388695:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845036.388781:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641641472:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 359us (555us total) trans 4294967734 rc 0/0 00000100:00100000:1.0:1788845036.388799:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 804 00000100:00100000:1.0:1788845036.392773:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.392827:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.392917:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641641728 00000100:00100000:3.0:1788845036.392927:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641641728 00000100:00100000:3.0:1788845036.392984:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 826 00000100:00100000:3.0:1788845036.393006:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641641728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.393025:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.393045:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ada04 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a912b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.393111:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660308352:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.393196:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.394015:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641641600 00000100:00100000:2.0:1788845036.394022:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641641600 00000100:00100000:2.0:1788845036.394066:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1201 00000100:00100000:2.0:1788845036.394073:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641641600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.394088:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.394103:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ada0b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9124 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.394133:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.394165:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660308480:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.394228:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.398110:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660308480:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845036.398140:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:2.0:1788845036.398159:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ada0b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9124 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.398182:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845036.398186:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845036.398191:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ada0b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9124 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.398205:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845036.398272:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641641600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4193us (5439us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845036.398282:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1201 00000100:00100000:1.0:1788845036.399727:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660308352:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.399794:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ada04 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a912b expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.399953:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000005d8ca32/0xb2187301287ada04 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a912b expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.399966:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e200, rc 0) 00000100:00100000:1.0:1788845036.400045:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641641728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7052us (7282us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.400052:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 826 00000100:00100000:1.0:1788845036.404128:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.404646:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.404767:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641641984 00000100:00100000:1.0:1788845036.404777:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641641984 00000100:00100000:1.0:1788845036.404845:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 827 00000100:00100000:1.0:1788845036.404856:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641641984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.404878:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.404942:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ada12 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9139 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.405005:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660308608:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.405010:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641641856 00000100:00100000:2.0:1788845036.405016:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641641856 00000100:00100000:2.0:1788845036.405060:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1202 00000100:00100000:2.0:1788845036.405069:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641641856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.405080:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1788845036.405083:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788845036.405096:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ada19 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9132 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.405128:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.405156:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660308736:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.405185:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.408687:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660308608:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845036.408728:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ada12 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9139 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.408731:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660308736:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845036.408751:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:0.0:1788845036.408758:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ada12 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9139 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.408767:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ada19 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9132 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.408774:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00010000:00010000:2.0:1788845036.408783:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845036.408798:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845036.408804:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ada19 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9132 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.408821:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845036.408852:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641641856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3783us (4737us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.408854:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641641984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3996us (4205us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845036.408871:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1202 00000100:00100000:0.0:1788845036.408871:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 827 00000100:00100000:0.0:1788845036.411205:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.411283:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641642112 00000100:00100000:1.0:1788845036.411294:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641642112 00000100:00100000:1.0:1788845036.411354:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1203 00000100:00100000:1.0:1788845036.411365:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641642112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.411407:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.411428:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ada20 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9140 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.411467:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.411507:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660308864:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.411595:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.411630:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.411729:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641642240 00000100:00100000:0.0:1788845036.411734:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641642240 00000100:00100000:0.0:1788845036.411773:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 828 00000100:00100000:0.0:1788845036.411781:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641642240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.411800:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.411811:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ada27 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9147 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.411844:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f23480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660308992:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.411913:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.414383:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f23480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660308992:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845036.414424:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ada27 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9147 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.414447:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ada27 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9147 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.414460:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:0.0:1788845036.414515:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641642240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2734us (2886us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.414524:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 828 00000100:00100000:1.0:1788845036.414614:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660308864:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.414642:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:1.0:1788845036.414656:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ada20 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9140 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.414670:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.414674:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.414678:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ada20 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9140 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.414689:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845036.414727:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641642112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3364us (3533us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.414734:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1203 00000100:00100000:3.0:1788845036.417940:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.418073:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641642368 00000100:00100000:1.0:1788845036.418083:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641642368 00000100:00100000:1.0:1788845036.418152:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1204 00000100:00100000:1.0:1788845036.418163:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641642368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.418201:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.418221:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada2e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a914e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.418258:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.418296:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660309120:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.418485:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.420175:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.420244:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641642496 00000100:00100000:3.0:1788845036.420252:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641642496 00000100:00100000:3.0:1788845036.420330:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 829 00000100:00100000:3.0:1788845036.420341:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641642496:12345-192.168.201.29@tcp:101:fsx.0 00000100:00100000:1.0:1788845036.420358:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660309120:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.420360:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.420379:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ada35 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9155 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845036.420384:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:1.0:1788845036.420404:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada2e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a914e expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.420438:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660309248:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.420513:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00010000:00010000:1.0:1788845036.420608:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.420612:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.420618:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada2e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a914e expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.420643:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845036.420684:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641642368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2523us (2753us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.420692:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1204 00000100:00100000:3.0:1788845036.422752:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660309248:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.422798:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ada35 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9155 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.422823:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ada35 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9155 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.422837:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845036.422934:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641642496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2594us (2764us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.422942:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 829 00000100:00100000:2.0:1788845036.426352:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.426581:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641642624 00000100:00100000:1.0:1788845036.426592:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641642624 00002000:00100000:1.0:1788845036.426651:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be5880 x1875739641642624/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.426678:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be5880 with x1875739641642624 00000100:00100000:1.0:1788845036.426709:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 805 00000100:00100000:1.0:1788845036.426717:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641642624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.426982:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641642624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 263us (641us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.426990:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 805 00002000:00100000:1.0:1788845036.426997:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be5880 x1875739641642624/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.427005:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be5880 with x1875739641642624 00000100:00100000:2.0:1788845036.431533:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.431597:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641642752 00000100:00100000:1.0:1788845036.431608:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641642752 00002000:00100000:1.0:1788845036.431671:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be5180 x1875739641642752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.431682:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be5180 with x1875739641642752 00000100:00100000:1.0:1788845036.431720:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 806 00000100:00100000:1.0:1788845036.431732:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641642752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.432244:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641642752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 512us (719us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.432253:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 806 00002000:00100000:1.0:1788845036.432261:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be5180 x1875739641642752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.432270:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be5180 with x1875739641642752 00000100:00100000:2.0:1788845036.435965:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.436056:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641642880 00000100:00100000:0.0:1788845036.436067:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641642880 00000100:00100000:0.0:1788845036.436134:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1205 00000100:00100000:0.0:1788845036.436145:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641642880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.436165:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.436187:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ada3c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a915c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.436230:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.436273:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f23480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660309376:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.436367:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.438927:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.439044:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641643008 00000100:00100000:1.0:1788845036.439067:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641643008 00000100:00100000:1.0:1788845036.439129:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 830 00000100:00100000:1.0:1788845036.439140:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641643008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.439158:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.439176:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ada43 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9163 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.439237:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660309504:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.439373:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.439592:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f23480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660309376:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.439614:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:0.0:1788845036.439630:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ada3c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a915c expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.439649:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.439652:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.439657:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ada3c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a915c expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.439681:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:0.0:1788845036.439713:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641642880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3570us (3756us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.439720:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1205 00000100:00100000:1.0:1788845036.442792:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660309504:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.442835:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ada43 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9163 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.442861:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ada43 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9163 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.442874:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845036.443063:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641643008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3924us (4179us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.443071:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 830 00000100:00100000:0.0:1788845036.446234:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.446444:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.446513:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641643264 00000100:00100000:1.0:1788845036.446523:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641643264 00000100:00100000:1.0:1788845036.446585:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 831 00000100:00100000:1.0:1788845036.446597:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641643264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.446617:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.446649:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ada4a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9171 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.446725:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660309632:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.446823:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.449363:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660309632:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.449410:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ada4a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9171 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.449437:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ada4a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9171 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.449451:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845036.449514:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641643264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2918us (3069us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.449523:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 831 00000100:00100000:2.0:1788845036.451191:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641643136 00000100:00100000:2.0:1788845036.451203:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641643136 00000100:00100000:2.0:1788845036.451295:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1206 00000100:00100000:2.0:1788845036.451312:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641643136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.451331:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.451350:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ada51 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a916a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.451384:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.451426:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660309760:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.451498:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.453713:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660309760:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845036.453737:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:2.0:1788845036.453755:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ada51 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a916a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.453776:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845036.453779:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845036.453795:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ada51 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a916a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.453810:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032800, rc 0) 00000100:00100000:2.0:1788845036.453946:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641643136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2642us (7720us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845036.453955:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1206 00000100:00100000:0.0:1788845036.457818:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.457921:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641643392 00000100:00100000:2.0:1788845036.457931:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641643392 00002000:00100000:2.0:1788845036.457979:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f21f80 x1875739641643392/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.457990:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f21f80 with x1875739641643392 00000100:00100000:2.0:1788845036.458022:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 807 00000100:00100000:2.0:1788845036.458032:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641643392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845036.458287:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641643392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 255us (476us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845036.458295:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 807 00002000:00100000:2.0:1788845036.458810:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f21f80 x1875739641643392/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.458822:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f21f80 with x1875739641643392 00000100:00100000:0.0:1788845036.462575:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.462707:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641643520 00000100:00100000:3.0:1788845036.462717:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641643520 00002000:00100000:3.0:1788845036.462766:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684380 x1875739641643520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.462777:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684380 with x1875739641643520 00000100:00100000:3.0:1788845036.462819:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 808 00000100:00100000:3.0:1788845036.462829:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641643520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.463124:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641643520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 296us (560us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.463132:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 808 00002000:00100000:3.0:1788845036.463139:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684380 x1875739641643520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.463146:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684380 with x1875739641643520 00000100:00100000:0.0:1788845036.466376:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.466465:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641643648 00000100:00100000:1.0:1788845036.466474:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641643648 00002000:00100000:1.0:1788845036.466521:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684700 x1875739641643648/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.466533:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684700 with x1875739641643648 00000100:00100000:1.0:1788845036.466569:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 809 00000100:00100000:1.0:1788845036.466579:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641643648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.466838:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641643648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (470us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.466874:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 809 00002000:00100000:1.0:1788845036.466914:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009684700 x1875739641643648/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.466923:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009684700 with x1875739641643648 00000100:00100000:0.0:1788845036.470549:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.470635:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641643776 00000100:00100000:3.0:1788845036.470645:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641643776 00002000:00100000:3.0:1788845036.470700:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009687100 x1875739641643776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.470712:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009687100 with x1875739641643776 00000100:00100000:3.0:1788845036.470750:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 810 00000100:00100000:3.0:1788845036.470760:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641643776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.471229:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641643776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 469us (689us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.471236:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 810 00002000:00100000:3.0:1788845036.471244:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009687100 x1875739641643776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.471252:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009687100 with x1875739641643776 00000100:00100000:0.0:1788845036.476077:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.476191:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641643904 00000100:00100000:1.0:1788845036.476202:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641643904 00002000:00100000:1.0:1788845036.476256:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009685c00 x1875739641643904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.476268:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009685c00 with x1875739641643904 00000100:00100000:2.0:1788845036.476418:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 811 00000100:00100000:2.0:1788845036.476432:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641643904:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845036.476705:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641643904:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 275us (638us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845036.476712:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 811 00002000:00100000:2.0:1788845036.476719:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009685c00 x1875739641643904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.476727:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009685c00 with x1875739641643904 00000100:00100000:0.0:1788845036.480476:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.480556:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641644032 00000100:00100000:3.0:1788845036.480565:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641644032 00002000:00100000:3.0:1788845036.480608:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009686d80 x1875739641644032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.480619:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009686d80 with x1875739641644032 00000100:00100000:3.0:1788845036.480651:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 812 00000100:00100000:3.0:1788845036.480660:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641644032:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.480940:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641644032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (473us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.480948:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 812 00002000:00100000:3.0:1788845036.480955:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009686d80 x1875739641644032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.480969:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009686d80 with x1875739641644032 00000100:00100000:0.0:1788845036.483489:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.483598:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641644160 00000100:00100000:2.0:1788845036.483608:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641644160 00002000:00100000:2.0:1788845036.483658:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009685180 x1875739641644160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.483678:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009685180 with x1875739641644160 00000100:00100000:2.0:1788845036.483714:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 813 00000100:00100000:2.0:1788845036.483723:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641644160:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845036.484010:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641644160:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 287us (528us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845036.484018:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 813 00002000:00100000:2.0:1788845036.484025:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009685180 x1875739641644160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.484032:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009685180 with x1875739641644160 00000100:00100000:0.0:1788845036.487575:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.487679:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641644288 00000100:00100000:3.0:1788845036.487689:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641644288 00002000:00100000:3.0:1788845036.487742:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009685880 x1875739641644288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.487753:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009685880 with x1875739641644288 00000100:00100000:3.0:1788845036.487802:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 814 00000100:00100000:3.0:1788845036.487812:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641644288:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.488095:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641644288:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 284us (530us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.488103:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 814 00002000:00100000:3.0:1788845036.488109:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009685880 x1875739641644288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.488117:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009685880 with x1875739641644288 00000100:00100000:1.0:1788845036.491176:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.491414:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641644416 00000100:00100000:2.0:1788845036.491424:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641644416 00002000:00100000:2.0:1788845036.491475:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8380 x1875739641644416/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.491492:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8380 with x1875739641644416 00000100:00100000:2.0:1788845036.491533:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 815 00000100:00100000:2.0:1788845036.491544:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641644416:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845036.491989:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641644416:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 445us (821us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845036.491997:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 815 00002000:00100000:2.0:1788845036.492005:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8380 x1875739641644416/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.492014:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8380 with x1875739641644416 00000100:00100000:0.0:1788845036.498854:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.499028:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641644544 00000100:00100000:3.0:1788845036.499038:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641644544 00002000:00100000:3.0:1788845036.499094:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009687800 x1875739641644544/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.499105:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009687800 with x1875739641644544 00000100:00100000:3.0:1788845036.499143:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 816 00000100:00100000:3.0:1788845036.499154:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641644544:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.499441:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641644544:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 287us (596us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.499453:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 816 00002000:00100000:3.0:1788845036.499461:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a009687800 x1875739641644544/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.499469:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a009687800 with x1875739641644544 00000100:00100000:0.0:1788845036.504781:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.504849:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.504957:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641644672 00000100:00100000:1.0:1788845036.504968:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641644672 00000100:00100000:1.0:1788845036.505032:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1207 00000100:00100000:1.0:1788845036.505043:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641644672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.505064:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.505085:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada58 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9178 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.505124:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.505166:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660309888:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.505254:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.505508:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641644800 00000100:00100000:2.0:1788845036.505513:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641644800 00000100:00100000:2.0:1788845036.505550:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 832 00000100:00100000:2.0:1788845036.505557:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641644800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.505567:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.505578:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287ada5f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a917f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.505615:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660310016:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.505658:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.508991:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660309888:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.509021:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:1.0:1788845036.509058:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada58 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9178 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.509080:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.509083:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.509089:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada58 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9178 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.509106:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845036.509166:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641644672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4124us (4394us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.509175:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1207 00000100:00100000:2.0:1788845036.509728:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660310016:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845036.509753:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287ada5f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a917f expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.509779:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287ada5f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a917f expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.509802:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032800, rc 0) 00000100:00100000:2.0:1788845036.509844:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641644800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4287us (4994us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845036.509852:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 832 00000100:00100000:0.0:1788845036.513563:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.513633:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641644928 00000100:00100000:1.0:1788845036.513642:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641644928 00000100:00100000:1.0:1788845036.513706:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1208 00000100:00100000:1.0:1788845036.513715:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641644928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.513736:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.513758:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ada66 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9186 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.513806:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.513844:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660310144:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.514163:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.516756:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.516948:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641645056 00000100:00100000:3.0:1788845036.516957:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641645056 00000100:00100000:3.0:1788845036.517005:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 833 00000100:00100000:3.0:1788845036.517015:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641645056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.517034:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.517053:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ada6d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a918d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.517118:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660310272:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.517197:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.517473:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660310144:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.517494:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:1.0:1788845036.517521:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ada66 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9186 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.517541:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.517544:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.517549:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ada66 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9186 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.517565:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845036.517610:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641644928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3896us (4057us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.517618:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1208 00000100:00100000:3.0:1788845036.521461:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660310272:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.521503:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ada6d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a918d expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.521528:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287ada6d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a918d expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.521541:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845036.521608:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641645056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4593us (4860us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.521621:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 833 00000100:00100000:2.0:1788845036.525358:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.525403:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.525495:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641645312 00000100:00100000:1.0:1788845036.525504:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641645312 00000100:00100000:1.0:1788845036.525564:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 834 00000100:00100000:1.0:1788845036.525576:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641645312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.525596:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.525613:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ada74 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a919b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.525682:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660310400:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.525772:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.526429:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641645184 00000100:00100000:0.0:1788845036.526434:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641645184 00000100:00100000:0.0:1788845036.526480:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1209 00000100:00100000:0.0:1788845036.526488:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641645184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.526500:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.526514:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287ada7b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9194 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.526540:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.526565:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660310528:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.526615:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.528858:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660310400:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.528935:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ada74 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a919b expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.528982:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ada74 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a919b expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.528997:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845036.529054:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641645312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3480us (3650us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.529063:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 834 00000100:00100000:0.0:1788845036.529344:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660310528:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.529360:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:0.0:1788845036.529373:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287ada7b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9194 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.529392:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.529396:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.529401:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287ada7b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9194 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.529412:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031a00, rc 0) 00000100:00100000:0.0:1788845036.529454:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641645184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2966us (4105us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.529461:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1209 00000100:00100000:2.0:1788845036.537036:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.537088:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.537186:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641645568 00000100:00100000:1.0:1788845036.537195:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641645568 00000100:00100000:1.0:1788845036.537265:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 835 00000100:00100000:1.0:1788845036.537276:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641645568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.537299:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.537328:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ada82 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91a9 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.537389:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660310656:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.537477:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.540239:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641645440 00000100:00100000:0.0:1788845036.540250:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641645440 00000100:00100000:0.0:1788845036.540316:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1210 00000100:00100000:0.0:1788845036.540327:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641645440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.540348:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.540370:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ada89 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91a2 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.540408:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.540446:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660310784:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.540537:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.541229:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660310656:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.541286:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ada82 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91a9 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.541322:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ada82 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91a9 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.541356:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845036.541407:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641645568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4133us (4318us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.541416:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 835 00000100:00100000:1.0:1788845036.544499:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660310784:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.544526:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:1.0:1788845036.544546:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ada89 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91a2 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.544568:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.544571:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.544576:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ada89 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91a2 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.544589:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:1.0:1788845036.544655:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641645440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4329us (7628us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.544671:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1210 00000100:00100000:3.0:1788845036.547700:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.547817:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.547926:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641645696 00000100:00100000:1.0:1788845036.547937:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641645696 00000100:00100000:1.0:1788845036.547993:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1211 00000100:00100000:1.0:1788845036.548002:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641645696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.548033:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.548053:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada90 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91b0 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.548113:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.548160:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660310912:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.548242:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.548800:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641645824 00000100:00100000:0.0:1788845036.548806:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641645824 00000100:00100000:0.0:1788845036.548899:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 836 00000100:00100000:0.0:1788845036.548908:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641645824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.548922:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.548934:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ada97 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91b7 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.548975:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660311040:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.549010:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.551842:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660311040:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.551905:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ada97 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91b7 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.551932:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ada97 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91b7 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.551946:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845036.552006:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641645824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3098us (4318us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.552015:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 836 00000100:00100000:2.0:1788845036.552476:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660310912:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845036.552494:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:2.0:1788845036.552506:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada90 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91b0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.552520:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845036.552523:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845036.552527:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ada90 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91b0 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.552539:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:2.0:1788845036.552568:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641645696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4566us (4750us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845036.552579:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1211 00000100:00100000:0.0:1788845036.557250:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.557337:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641645952 00000100:00100000:3.0:1788845036.557346:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641645952 00000100:00100000:3.0:1788845036.557403:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1212 00000100:00100000:3.0:1788845036.557412:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641645952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.557430:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.557450:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287ada9e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91be expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.557485:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.557535:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660311168:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.557617:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.560126:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.560200:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641646080 00000100:00100000:3.0:1788845036.560209:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641646080 00000100:00100000:3.0:1788845036.560273:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 837 00000100:00100000:3.0:1788845036.560283:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641646080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.560310:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.560350:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287adaa5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91c5 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.560409:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660311296:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.560492:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.562152:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660311168:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845036.562176:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:3.0:1788845036.562195:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287ada9e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91be expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.562214:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.562217:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845036.562222:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f400/0xb2187301287ada9e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91be expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.562236:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f400, rc 0) 00000100:00100000:3.0:1788845036.562298:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641645952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4887us (5056us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.562319:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1212 00000100:00100000:3.0:1788845036.564208:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660311296:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.564243:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287adaa5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91c5 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.564280:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287adaa5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91c5 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.564293:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845036.564347:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641646080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4065us (4229us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.564355:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 837 00000100:00100000:0.0:1788845036.568219:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.568403:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641646208 00000100:00100000:3.0:1788845036.568413:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641646208 00000100:00100000:3.0:1788845036.568475:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1213 00000100:00100000:3.0:1788845036.568485:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641646208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.568506:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.568527:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91cc expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.568558:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.568567:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.568607:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.568624:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 14 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.568638:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660311424:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845036.568727:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.572173:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.572315:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660311424:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845036.572354:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.572375:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.572378:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845036.572383:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.572394:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e600, rc 0) 00000100:00100000:3.0:1788845036.572460:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641646208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3976us (4250us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845036.572469:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1213 00000100:00100000:1.0:1788845036.573908:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641646336 00000100:00100000:1.0:1788845036.573920:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641646336 00002000:00100000:1.0:1788845036.573969:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023df800 x1875739641646336/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.573980:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023df800 with x1875739641646336 00010000:00010000:1.0:1788845036.573988:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 15 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.574008:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 15 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845036.574016:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f023df800 00000100:00100000:1.0:1788845036.574096:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 28 00000100:00100000:1.0:1788845036.574106:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641646336:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845036.577530:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 371399 -> 371399 00000004:00010000:1.0:1788845036.577541:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.577544:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.577546:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.577548:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845036.577634:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641646336:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3523us (5464us total) trans 4294967735 rc 0/0 00000100:00100000:1.0:1788845036.577645:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 28 00002000:00100000:1.0:1788845036.577666:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023df800 x1875739641646336/t4294967735(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/448 e 0 to 0 dl 1788845047 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.577676:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023df800 with x1875739641646336 00010000:00010000:1.0:1788845036.577686:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 17 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.577706:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 17 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845036.577714:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f023df800 00000100:00100000:0.0:1788845036.581229:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.581435:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641646464 00000100:00100000:3.0:1788845036.581444:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641646464 00010000:00010000:3.0:1788845036.581486:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 17 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.581566:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 419 00000100:00100000:3.0:1788845036.581578:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641646464:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845036.581593:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845036.581622:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 17 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.581641:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 17 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.581663:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ad97f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90b4 expref: 17 pid: 5839 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.581688:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.581695:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.581707:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.581716:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.581754:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.581773:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660311552:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845036.581863:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.584161:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660311552:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845036.584192:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845036.584260:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641646464:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2685us (3042us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.584271:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 419 00000100:00100000:2.0:1788845036.590156:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.590236:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641646592 00000100:00100000:3.0:1788845036.590247:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641646592 00002000:00100000:3.0:1788845036.590317:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02382680 x1875739641646592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.590328:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02382680 with x1875739641646592 00000100:00100000:3.0:1788845036.590365:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 817 00000100:00100000:3.0:1788845036.590376:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641646592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.599104:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641646592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8718us (8947us total) trans 0 rc 372736/372736 00000100:00100000:1.0:1788845036.599121:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 817 00002000:00100000:1.0:1788845036.599136:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02382680 x1875739641646592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/440 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 372736/372736 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.599146:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02382680 with x1875739641646592 00000100:00100000:0.0:1788845036.602626:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.602762:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641646720 00000100:00100000:1.0:1788845036.602773:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641646720 00000100:00100000:1.0:1788845036.602859:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1214 00000100:00100000:1.0:1788845036.602870:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641646720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.602927:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.602953:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adab3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91d3 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.602995:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adab3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91d3 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845036.603008:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 371399 00010000:00010000:1.0:1788845036.603016:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adab3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91d3 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.603028:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.603031:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.603037:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845036.603099:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641646720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 231us (483us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.603106:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1214 00000100:00100000:0.0:1788845036.606431:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.606587:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641646976 00000100:00100000:2.0:1788845036.606597:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641646976 00000100:00100000:2.0:1788845036.606669:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1215 00000100:00100000:2.0:1788845036.606680:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641646976:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845036.606741:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845036.606770:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ad9b7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90de expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.606801:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845036.606807:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845036.607078:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.607094:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.607103:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.607139:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.607159:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 14 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.607174:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660311680:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845036.607249:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.610584:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.610944:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660311680:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.610973:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.611065:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641647104 00000100:00100000:3.0:1788845036.611072:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641647104 00010000:00010000:3.0:1788845036.611114:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 15 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.611156:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 420 00000100:00100000:3.0:1788845036.611167:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641647104:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845036.611183:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845036.611190:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 15 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.611209:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 15 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.611233:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ad9c5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90e5 expref: 15 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.611251:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.611256:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.611266:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.611272:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.611298:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845036.611361:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641647104:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 196us (786us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845036.611370:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 420 00010000:00010000:1.0:1788845036.611511:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.611519:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845036.611650:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967736 00000004:00010000:1.0:1788845036.611714:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 371399 -> 371399 00000004:00010000:1.0:1788845036.611718:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.611721:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.611723:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.611726:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:1.0:1788845036.611733:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008ae1970d reply state 000000000b80b9a1 transno 4294967737 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.611743:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845036.611823:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:14891:x1875739641646976:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5144us (5406us total) trans 4294967737 rc 0/0 00000100:00100000:1.0:1788845036.611830:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1215 00000020:00080000:1.0:1788845036.611833:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845036.614473:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.614489:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.614497:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.614503:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02032200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845036.614508:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.614517:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.614558:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.614562:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845036.614566:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.614573:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287adaba lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.616284:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.616352:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641647232 00000100:00100000:1.0:1788845036.616361:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641647232 00000100:00100000:1.0:1788845036.616425:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1216 00000100:00100000:1.0:1788845036.616436:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641647232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.616463:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.616466:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788845036.616509:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adab3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91d3 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.616528:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.616532:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845036.616546:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91da expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.616569:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.616579:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.616631:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.616649:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 13 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.616670:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660311808:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845036.616754:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.620962:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.621091:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660311808:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.621127:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.621146:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.621149:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.621154:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.621165:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845036.621220:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641647232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4786us (4945us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.621228:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1216 00000100:00100000:2.0:1788845036.621330:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641647360 00000100:00100000:2.0:1788845036.621337:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641647360 00010000:00010000:2.0:1788845036.621367:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 14 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.621402:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 421 00000100:00100000:2.0:1788845036.621411:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641647360:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845036.621426:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845036.621452:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 14 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.621469:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 14 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.621492:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e600/0xb2187301287adaac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91cc expref: 14 pid: 5841 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.621506:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845036.621511:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.621520:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.621536:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.621568:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.621580:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02381f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660311936:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845036.621631:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.626509:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.627106:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02381f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660311936:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845036.627132:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845036.627189:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641647360:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5779us (6238us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845036.627200:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 421 00000100:00100000:1.0:1788845036.627284:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641647488 00000100:00100000:1.0:1788845036.627294:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641647488 00002000:00010000:1.0:1788845036.627344:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845036.627350:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023ddc00 with x1875739641647488 00000100:00100000:1.0:1788845036.627409:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 818 00000100:00100000:1.0:1788845036.627419:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641647488:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845036.627615:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 371399 -> 453107 00000004:00010000:1.0:1788845036.627619:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.627622:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.627624:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.627627:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845036.627672:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641647488:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 255us (1173us total) trans 4294967738 rc 0/0 00000100:00100000:1.0:1788845036.627679:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 818 00002000:00010000:1.0:1788845036.627683:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845036.627685:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023ddc00 with x1875739641647488 00000100:00100000:0.0:1788845036.629685:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.629751:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641647616 00000100:00100000:1.0:1788845036.629761:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641647616 00002000:00010000:1.0:1788845036.629803:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845036.629807:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dc000 with x1875739641647616 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845036.629818:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dc000. 00000100:00100000:1.0:1788845036.629850:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 425 00000100:00100000:1.0:1788845036.629860:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641647616:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845036.630107:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1788845036.630110:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00002000:00010000:1.0:1788845036.630113:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845036 -> 1788845035 00002000:00010000:1.0:1788845036.630114:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00002000:00010000:1.0:1788845036.630116:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:1.0:1788845036.630176:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641647616:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 316us (499us total) trans 4294967443 rc 0/0 00000100:00100000:1.0:1788845036.630197:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 425 00002000:00010000:1.0:1788845036.630201:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845036.630204:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dc000 with x1875739641647616 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845036.630209:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dc000. 00000100:00100000:0.0:1788845036.634060:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.634167:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641647744 00000100:00100000:1.0:1788845036.634179:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641647744 00002000:00100000:1.0:1788845036.634236:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dfb80 x1875739641647744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.634247:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dfb80 with x1875739641647744 00000100:00100000:1.0:1788845036.634286:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 819 00000100:00100000:1.0:1788845036.634295:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2368:x1875739641647744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.634583:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2368:x1875739641647744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 287us (533us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.634590:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 819 00002000:00100000:1.0:1788845036.634598:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dfb80 x1875739641647744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.634605:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dfb80 with x1875739641647744 00000100:00100000:0.0:1788845036.638429:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.638575:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641647872 00000100:00100000:1.0:1788845036.638586:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641647872 00002000:00100000:1.0:1788845036.638636:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dc380 x1875739641647872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.638647:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dc380 with x1875739641647872 00000100:00100000:1.0:1788845036.638696:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 820 00000100:00100000:1.0:1788845036.638708:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2367:x1875739641647872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.639031:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2367:x1875739641647872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 324us (615us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.639040:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 820 00002000:00100000:1.0:1788845036.639048:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dc380 x1875739641647872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.639056:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dc380 with x1875739641647872 00000100:00100000:0.0:1788845036.643358:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.643616:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641648000 00000100:00100000:1.0:1788845036.643627:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641648000 00002000:00100000:1.0:1788845036.643689:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dd500 x1875739641648000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.643700:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dd500 with x1875739641648000 00000100:00100000:1.0:1788845036.643737:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 821 00000100:00100000:1.0:1788845036.643747:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641648000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.644070:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641648000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 323us (724us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.644077:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 821 00002000:00100000:1.0:1788845036.644084:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dd500 x1875739641648000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.644091:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dd500 with x1875739641648000 00000100:00100000:0.0:1788845036.647630:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.647710:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641648128 00000100:00100000:2.0:1788845036.647722:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641648128 00002000:00100000:2.0:1788845036.647789:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dce00 x1875739641648128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.647801:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dce00 with x1875739641648128 00000100:00100000:2.0:1788845036.647844:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 822 00000100:00100000:2.0:1788845036.647856:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641648128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845036.648168:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641648128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 313us (553us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845036.648177:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 822 00002000:00100000:2.0:1788845036.648185:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dce00 x1875739641648128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845036.648193:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dce00 with x1875739641648128 00000100:00100000:0.0:1788845036.652872:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.653001:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641648256 00000100:00100000:1.0:1788845036.653011:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641648256 00002000:00100000:1.0:1788845036.653062:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849880 x1875739641648256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.653074:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849880 with x1875739641648256 00000100:00100000:1.0:1788845036.653128:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 823 00000100:00100000:1.0:1788845036.653139:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2367:x1875739641648256:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.653418:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2367:x1875739641648256:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (556us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.653425:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 823 00002000:00100000:1.0:1788845036.653433:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849880 x1875739641648256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.653441:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849880 with x1875739641648256 00000100:00100000:0.0:1788845036.657116:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.659948:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641648384 00000100:00100000:1.0:1788845036.659964:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641648384 00002000:00100000:1.0:1788845036.660026:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848380 x1875739641648384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.660038:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848380 with x1875739641648384 00000100:00100000:1.0:1788845036.660082:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 824 00000100:00100000:1.0:1788845036.660092:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2367:x1875739641648384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.660356:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2367:x1875739641648384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 265us (3256us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.660365:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 824 00002000:00100000:1.0:1788845036.660372:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848380 x1875739641648384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.660380:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848380 with x1875739641648384 00000100:00100000:0.0:1788845036.664542:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.664649:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641648512 00000100:00100000:1.0:1788845036.664667:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641648512 00002000:00100000:1.0:1788845036.664718:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849500 x1875739641648512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.664751:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849500 with x1875739641648512 00000100:00100000:1.0:1788845036.664802:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 825 00000100:00100000:1.0:1788845036.664813:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641648512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.665270:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641648512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 457us (735us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.665278:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 825 00002000:00100000:1.0:1788845036.665285:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849500 x1875739641648512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.665293:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849500 with x1875739641648512 00000100:00100000:0.0:1788845036.668801:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.669151:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641648640 00000100:00100000:1.0:1788845036.669162:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641648640 00002000:00100000:1.0:1788845036.669214:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848700 x1875739641648640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.669225:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848700 with x1875739641648640 00000100:00100000:1.0:1788845036.669265:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 826 00000100:00100000:1.0:1788845036.669274:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641648640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.669960:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641648640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 683us (1177us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.669969:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 826 00002000:00100000:1.0:1788845036.669978:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848700 x1875739641648640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.669986:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848700 with x1875739641648640 00000100:00100000:2.0:1788845036.674348:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.674431:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641648768 00000100:00100000:3.0:1788845036.674441:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641648768 00002000:00100000:3.0:1788845036.674495:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02381f80 x1875739641648768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.674507:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02381f80 with x1875739641648768 00000100:00100000:3.0:1788845036.674547:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 827 00000100:00100000:3.0:1788845036.674557:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641648768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.674832:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641648768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 275us (492us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.674840:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 827 00002000:00100000:3.0:1788845036.674847:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02381f80 x1875739641648768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.674859:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02381f80 with x1875739641648768 00000100:00100000:0.0:1788845036.678230:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.678340:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641648896 00000100:00100000:1.0:1788845036.678364:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641648896 00002000:00100000:1.0:1788845036.678414:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384ad80 x1875739641648896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.678426:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384ad80 with x1875739641648896 00000100:00100000:1.0:1788845036.678467:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 828 00000100:00100000:1.0:1788845036.678476:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641648896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.678764:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641648896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 288us (545us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.678772:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 828 00002000:00100000:1.0:1788845036.678788:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384ad80 x1875739641648896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.678796:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384ad80 with x1875739641648896 00000100:00100000:2.0:1788845036.682526:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.682633:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641649024 00000100:00100000:3.0:1788845036.682646:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641649024 00002000:00100000:3.0:1788845036.682713:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380380 x1875739641649024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.682725:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380380 with x1875739641649024 00000100:00100000:3.0:1788845036.682766:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 829 00000100:00100000:3.0:1788845036.682776:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641649024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845036.683111:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641649024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 335us (599us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845036.683120:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 829 00002000:00100000:3.0:1788845036.683128:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02380380 x1875739641649024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.683136:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02380380 with x1875739641649024 00000100:00100000:0.0:1788845036.687001:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.687088:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641649152 00000100:00100000:1.0:1788845036.687097:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641649152 00002000:00100000:1.0:1788845036.687151:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384aa00 x1875739641649152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.687162:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384aa00 with x1875739641649152 00000100:00100000:1.0:1788845036.687202:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 830 00000100:00100000:1.0:1788845036.687212:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641649152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.687710:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2370:x1875739641649152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 496us (719us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.687720:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 830 00002000:00100000:1.0:1788845036.687728:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384aa00 x1875739641649152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.687759:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384aa00 with x1875739641649152 00000100:00100000:0.0:1788845036.692009:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.692104:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641649280 00000100:00100000:3.0:1788845036.692114:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641649280 00002000:00100000:3.0:1788845036.692164:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384b100 x1875739641649280/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.692176:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384b100 with x1875739641649280 00000100:00100000:3.0:1788845036.692218:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 831 00000100:00100000:3.0:1788845036.692228:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641649280:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845036.695552:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 453107 -> 453107 00000004:00010000:3.0:1788845036.695561:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:3.0:1788845036.695565:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:3.0:1788845036.695568:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:3.0:1788845036.695570:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845036.695705:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2369:x1875739641649280:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3472us (3701us total) trans 4294967739 rc 0/0 00000100:00100000:3.0:1788845036.695718:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 831 00002000:00100000:3.0:1788845036.695731:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384b100 x1875739641649280/t4294967739(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/448 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845036.695742:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384b100 with x1875739641649280 00000100:00100000:0.0:1788845036.699505:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.699601:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641649408 00000100:00100000:3.0:1788845036.699611:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641649408 00000100:00100000:3.0:1788845036.699720:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1217 00000100:00100000:3.0:1788845036.699730:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641649408:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:0.0:1788845036.724389:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641649408:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 24646us (24880us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.724405:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1217 00000001:00080000:2.0:1788845036.726916:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967731 is committed 00000001:00080000:2.0:1788845036.726937:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967738 is committed 00000001:00080000:2.0:1788845036.726964:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967739 is committed 00000100:00100000:2.0:1788845036.727397:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.727496:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641649536 00000100:00100000:3.0:1788845036.727507:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641649536 00000100:00100000:3.0:1788845036.727616:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1218 00000100:00100000:3.0:1788845036.727628:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641649536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.727650:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.727740:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287adac8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91e1 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.727839:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287adacf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.727854:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287adacf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.727870:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287adacf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845036.727916:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 453107 00000004:00200000:3.0:1788845036.727920:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845036.727940:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287adacf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.727965:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287adac8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91e1 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.727982:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287adacf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91e1 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.727991:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.727993:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845036.727996:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ec00, rc 0) 00000100:00100000:3.0:1788845036.728052:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641649536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 426us (659us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845036.728058:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1218 00000100:00100000:2.0:1788845036.732198:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.732279:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641649792 00000100:00100000:1.0:1788845036.732289:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641649792 00000100:00100000:1.0:1788845036.732355:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1219 00000100:00100000:1.0:1788845036.732365:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641649792:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845036.732419:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845036.732449:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ec00/0xb2187301287adacf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91e1 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.732468:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.732474:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845036.732875:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.732923:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.732934:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845036.732987:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967740 00000004:00010000:1.0:1788845036.733012:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 453107 -> 453107 00000004:00010000:1.0:1788845036.733016:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.733019:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.733021:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.733024:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 648 00000004:00010000:1.0:1788845036.733030:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b912c8af reply state 0000000007dbb5f5 transno 4294967741 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.733052:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845036.733199:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641649792:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 834us (1007us total) trans 4294967741 rc 0/0 00000100:00100000:1.0:1788845036.733207:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1219 00000020:00080000:1.0:1788845036.733211:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845036.736796:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:3.0:1788845036.736998:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.737012:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.737020:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.737031:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845036.737036:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.737044:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.737062:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.737066:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845036.737070:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.737077:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adad6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.738833:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641649920 00000100:00100000:1.0:1788845036.738849:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641649920 00002000:00010000:1.0:1788845036.738918:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1127183->18446744073709551615) 00002000:00100000:1.0:1788845036.738923:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03849c00 with x1875739641649920 ext(1127183->18446744073709551615) 00002000:00010000:1.0:1788845036.738936:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03849c00. 00000100:00100000:1.0:1788845036.738973:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 426 00000100:00100000:1.0:1788845036.738983:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641649920:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845036.739226:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1127183 00002000:00010000:1.0:1788845036.739231:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00002000:00010000:1.0:1788845036.739233:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845035 -> 1788845035 00002000:00010000:1.0:1788845036.739236:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00002000:00010000:1.0:1788845036.739238:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:1.0:1788845036.739620:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641649920:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 636us (2840us total) trans 4294967444 rc 0/0 00000100:00100000:1.0:1788845036.739630:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 426 00002000:00010000:1.0:1788845036.739634:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1127183->18446744073709551615) 00002000:00100000:1.0:1788845036.739638:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03849c00 with x1875739641649920 ext(1127183->18446744073709551615) 00002000:00010000:1.0:1788845036.739643:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03849c00. 00000100:00100000:2.0:1788845036.742269:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.742414:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641650048 00000100:00100000:1.0:1788845036.742423:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641650048 00000100:00100000:1.0:1788845036.742522:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1220 00000100:00100000:1.0:1788845036.742533:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641650048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.742554:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.742575:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adadd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91e8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.742732:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adae4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.742748:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adae4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.742760:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adae4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845036.742797:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1127183 00000004:00200000:1.0:1788845036.742801:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845036.742818:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adae4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.742843:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adadd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91e8 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.742859:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adae4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91e8 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.742867:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.742870:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845036.742874:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:1.0:1788845036.743069:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641650048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 538us (808us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.743083:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1220 00000100:00100000:2.0:1788845036.745856:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.746054:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641650176 00000100:00100000:0.0:1788845036.746064:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641650176 00000100:00100000:0.0:1788845036.746120:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1221 00000100:00100000:0.0:1788845036.746130:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641650176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.746149:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.746169:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287adaeb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91ef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.746206:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.746245:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660312064:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.746341:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.748758:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.748839:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641650304 00000100:00100000:1.0:1788845036.748849:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641650304 00000100:00100000:1.0:1788845036.748939:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 838 00000100:00100000:1.0:1788845036.748950:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641650304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.748971:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.748989:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adaf2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91f6 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.749052:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660312192:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.749146:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845036.751485:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845035 -> 1788845036 00000100:00100000:1.0:1788845036.751516:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660312192:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.751554:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adaf2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91f6 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.751579:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adaf2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91f6 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.751592:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845036.751654:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641650304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2705us (2906us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.751669:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 838 00000100:00100000:0.0:1788845036.751861:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660312064:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.751878:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:0.0:1788845036.751916:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287adaeb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91ef expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.751932:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.751934:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.751939:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287adaeb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91ef expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.751950:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:0.0:1788845036.751975:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641650176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5847us (6133us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.751982:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1221 00000100:00100000:2.0:1788845036.755383:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.755465:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641650432 00000100:00100000:3.0:1788845036.755474:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641650432 00000100:00100000:3.0:1788845036.755527:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1222 00000100:00100000:3.0:1788845036.755536:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641650432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.755556:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.755574:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287adaf9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a91fd expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.755608:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.755647:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660312320:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.755731:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.757985:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.758108:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641650560 00000100:00100000:1.0:1788845036.758118:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641650560 00000100:00100000:1.0:1788845036.758184:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 839 00000100:00100000:1.0:1788845036.758194:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641650560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.758214:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.758232:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adb00 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9204 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.758294:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660312448:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.758380:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.758947:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660312320:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845036.758969:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:3.0:1788845036.758985:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287adaf9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91fd expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.759002:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.759005:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845036.759010:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e000/0xb2187301287adaf9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91fd expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.759024:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e000, rc 0) 00000100:00100000:3.0:1788845036.759063:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641650432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3528us (3689us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.759071:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1222 00000100:00100000:1.0:1788845036.762927:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660312448:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.762971:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adb00 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9204 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.762997:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adb00 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9204 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.763011:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845036.763062:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641650560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4869us (5086us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.763070:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 839 00000100:00100000:0.0:1788845036.766587:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.766632:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.766745:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641650688 00000100:00100000:1.0:1788845036.766755:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641650688 00000100:00100000:1.0:1788845036.766834:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1223 00000100:00100000:1.0:1788845036.766845:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641650688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.766865:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.766911:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adb07 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a920b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.766947:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.766987:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660312576:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.767061:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.767121:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641650816 00000100:00100000:3.0:1788845036.767125:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641650816 00000100:00100000:3.0:1788845036.767164:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 840 00000100:00100000:3.0:1788845036.767172:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641650816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.767182:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.767193:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287adb0e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9212 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.767229:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660312704:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.767257:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.770272:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660312576:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.770312:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:1.0:1788845036.770332:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adb07 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a920b expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.770354:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.770357:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.770363:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adb07 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a920b expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.770380:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845036.770440:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641650688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3596us (3861us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.770449:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1223 00000100:00100000:3.0:1788845036.770559:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660312704:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.770581:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287adb0e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9212 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.770600:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287adb0e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9212 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.770610:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e000, rc 0) 00000100:00100000:3.0:1788845036.770635:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641650816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3464us (4003us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.770642:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 840 00000100:00100000:2.0:1788845036.774292:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.774344:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.774423:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641650944 00000100:00100000:1.0:1788845036.774432:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641650944 00000100:00100000:1.0:1788845036.774495:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1224 00000100:00100000:1.0:1788845036.774505:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641650944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.774527:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.774548:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb15 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9219 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.774582:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.774621:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660312832:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.774714:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.774929:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641651072 00000100:00100000:3.0:1788845036.774933:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641651072 00000100:00100000:3.0:1788845036.774965:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 841 00000100:00100000:3.0:1788845036.774973:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641651072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.774982:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.774993:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8699c30/0xb2187301287adb1c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9220 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.775028:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660312960:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.775061:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.778972:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660312960:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845036.779015:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b8699c30/0xb2187301287adb1c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9220 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.779042:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8699c30/0xb2187301287adb1c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9220 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.779062:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e800, rc 0) 00000100:00100000:3.0:1788845036.779117:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641651072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4143us (4770us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.779127:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 841 00000100:00100000:1.0:1788845036.779255:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660312832:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.779271:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:1.0:1788845036.779284:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb15 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9219 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.779298:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.779302:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.779307:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb15 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9219 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.779318:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845036.779344:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641650944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4841us (5060us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.779352:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1224 00000100:00100000:0.0:1788845036.782680:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.782718:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.782810:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641651328 00000100:00100000:1.0:1788845036.782819:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641651328 00000100:00100000:1.0:1788845036.782875:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 842 00000100:00100000:1.0:1788845036.782918:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641651328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.782937:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.782955:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb23 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a922e expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.783012:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660313088:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.783079:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.783174:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641651200 00000100:00100000:2.0:1788845036.783178:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641651200 00000100:00100000:2.0:1788845036.783213:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1225 00000100:00100000:2.0:1788845036.783219:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641651200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.783229:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.783241:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287adb2a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9227 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.783268:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.783289:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02381500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660313216:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.783315:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.788789:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660313088:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.788830:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb23 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a922e expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.788855:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb23 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a922e expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.788867:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845036.788965:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641651328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6080us (6305us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.788974:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 842 00000100:00100000:0.0:1788845036.791084:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02381500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660313216:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.791113:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:0.0:1788845036.791133:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287adb2a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9227 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.791155:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.791158:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.791162:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287adb2a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9227 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.791178:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:0.0:1788845036.791234:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641651200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8013us (8514us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.791243:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1225 00000100:00100000:0.0:1788845036.795105:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.795175:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641651456 00000100:00100000:2.0:1788845036.795184:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641651456 00000100:00100000:2.0:1788845036.795237:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1226 00000100:00100000:2.0:1788845036.795245:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641651456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.795265:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.795284:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287adb31 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9235 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.795332:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.795373:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02381500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660313344:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.795460:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.797672:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02381500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660313344:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.797696:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:0.0:1788845036.797713:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287adb31 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9235 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.797733:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.797736:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.797741:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287adb31 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9235 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.797755:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:0.0:1788845036.797822:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641651456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2576us (2726us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.797830:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1226 00000100:00100000:3.0:1788845036.799605:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.799702:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641651584 00000100:00100000:1.0:1788845036.799711:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641651584 00000100:00100000:1.0:1788845036.799769:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 843 00000100:00100000:1.0:1788845036.799789:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641651584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.799807:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.799825:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb38 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a923c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.799924:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660313472:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.799996:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.802113:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660313472:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.802155:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb38 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a923c expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.802182:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb38 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a923c expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.802207:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845036.802261:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641651584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2483us (2663us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.802270:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 843 00000100:00100000:2.0:1788845036.810787:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.810835:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.810922:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641651712 00000100:00100000:1.0:1788845036.810931:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641651712 00002000:00100000:1.0:1788845036.810977:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383b80 x1875739641651712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 504/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.810989:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383b80 with x1875739641651712 00000100:00100000:1.0:1788845036.811021:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 832 00000100:00100000:1.0:1788845036.811030:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641651712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.817659:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641651712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 6612us (6875us total) trans 0 rc 405504/405504 00000100:00100000:1.0:1788845036.817673:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 832 00002000:00100000:1.0:1788845036.817687:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383b80 x1875739641651712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 504/440 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 405504/405504 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.817704:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383b80 with x1875739641651712 00000100:00100000:0.0:1788845036.817820:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641651840 00000100:00100000:0.0:1788845036.817829:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641651840 00002000:00100000:0.0:1788845036.817863:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1130495) req@ffff949f02381500 x1875739641651840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.817872:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02381500 with x1875739641651840 ext(1048576->1130495) 00002000:00010000:0.0:1788845036.817876:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02381500 00000100:00100000:0.0:1788845036.817920:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 427 00000100:00100000:0.0:1788845036.817928:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641651840:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845036.823673:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.824349:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641651968 00000100:00100000:1.0:1788845036.824360:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641651968 00000100:00100000:1.0:1788845036.824475:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1227 00000100:00100000:1.0:1788845036.824485:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641651968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.824508:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.824529:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adb3f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9243 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.824617:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.824632:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.824647:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845036.824843:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1127183 00000004:00200000:1.0:1788845036.824848:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845036.824868:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.824927:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adb3f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.824946:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.824959:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.824962:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845036.824965:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845036.825127:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641651968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 642us (1471us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.825134:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1227 00000100:00100000:0.0:1788845036.829238:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641651840:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 11298us (18389us total) trans 0 rc 81920/81920 00000100:00100000:0.0:1788845036.829253:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 427 00002000:00100000:0.0:1788845036.829265:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1130495) req@ffff949f02381500 x1875739641651840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:142/0 lens 488/440 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 81920/81920 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.829276:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02381500 with x1875739641651840 ext(1048576->1130495) 00002000:00010000:0.0:1788845036.829283:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02381500 00000100:00100000:2.0:1788845036.830306:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.830342:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.830814:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641652096 00000100:00100000:3.0:1788845036.830824:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641652096 00000100:00100000:3.0:1788845036.830875:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1228 00000100:00100000:3.0:1788845036.830902:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641652096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.830921:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.830939:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287adb4d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a924a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.830973:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.831010:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660313600:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.831070:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.831088:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641652224 00000100:00100000:3.0:1788845036.831091:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641652224 00000100:00100000:3.0:1788845036.831121:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 844 00000100:00100000:3.0:1788845036.831128:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641652224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.831135:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.831145:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287adb54 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9251 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.831180:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660313728:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.831200:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.834379:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660313728:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845036.834428:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287adb54 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9251 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.834455:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d0a6b26/0xb2187301287adb54 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9251 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.834468:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e000, rc 0) 00000100:00100000:0.0:1788845036.834532:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641652224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3402us (4187us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.834541:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 844 00000100:00100000:3.0:1788845036.834644:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660313600:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845036.834666:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:3.0:1788845036.834679:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287adb4d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a924a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.834694:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.834698:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845036.834703:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191e800/0xb2187301287adb4d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a924a expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.834714:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e800, rc 0) 00000100:00100000:3.0:1788845036.834738:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641652096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3856us (4448us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.834746:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1228 00000100:00100000:2.0:1788845036.838472:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.838555:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641652352 00000100:00100000:3.0:1788845036.838566:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641652352 00000100:00100000:3.0:1788845036.838629:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 845 00000100:00100000:3.0:1788845036.838640:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641652352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.838668:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.838688:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9258 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.838721:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.838731:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(6 8) 00010000:00010000:3.0:1788845036.838735:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:3.0:1788845036.838740:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1105920->1110015] (req 1105920->1110015) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 8 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.838776:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.838805:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 10 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.838819:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660313856:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845036.839012:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.842599:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.843798:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660313856:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845036.843833:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1105920->1110015] (req 1105920->1110015) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 8 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.843858:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fc00, rc 0) 00000100:00100000:0.0:1788845036.843933:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641652352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5294us (5473us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.843943:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 845 00000100:00100000:1.0:1788845036.845128:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641652480 00000100:00100000:1.0:1788845036.845143:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641652480 00010000:00010000:1.0:1788845036.845190:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 11 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.845294:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 422 00000100:00100000:1.0:1788845036.845315:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641652480:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845036.845331:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845036.845354:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 11 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.845376:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 11 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.845404:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f74df9f/0xb2187301287ad9a9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a90d7 expref: 11 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.845420:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:1.0:1788845036.845424:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:1.0:1788845036.845432:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.845442:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.845481:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.845495:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660313984:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845036.845576:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.849005:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.849110:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641652608 00000100:00100000:0.0:1788845036.849119:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641652608 00002000:00100000:0.0:1788845036.849168:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1105920->1110015) req@ffff94a03f6c7480 x1875739641652608/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.849178:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c7480 with x1875739641652608 ext(1105920->1110015) 00002000:00010000:0.0:1788845036.849184:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c7480 00000100:00100000:0.0:1788845036.849220:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 428 00000100:00100000:0.0:1788845036.849229:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641652608:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.849528:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641652608:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 298us (533us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.849535:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 428 00002000:00100000:0.0:1788845036.849542:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1105920->1110015) req@ffff94a03f6c7480 x1875739641652608/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.849551:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c7480 with x1875739641652608 ext(1105920->1110015) 00002000:00010000:0.0:1788845036.849554:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c7480 00000100:00100000:1.0:1788845036.850030:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660313984:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845036.850051:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845036.850094:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641652480:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4781us (7504us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.850104:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 422 00000100:00100000:3.0:1788845036.853030:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.853148:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641652736 00000100:00100000:0.0:1788845036.853158:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641652736 00002000:00100000:0.0:1788845036.853208:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1110016->1114111) req@ffff94a03f6c4000 x1875739641652736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.853219:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c4000 with x1875739641652736 ext(1110016->1114111) 00002000:00010000:0.0:1788845036.853224:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c4000 00000100:00100000:0.0:1788845036.853265:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 429 00000100:00100000:0.0:1788845036.853275:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641652736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.853531:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641652736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (509us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.853539:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 429 00002000:00100000:0.0:1788845036.853545:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1110016->1114111) req@ffff94a03f6c4000 x1875739641652736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.853553:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c4000 with x1875739641652736 ext(1110016->1114111) 00002000:00010000:0.0:1788845036.853557:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c4000 00000100:00100000:3.0:1788845036.858120:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.858408:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641652864 00000100:00100000:1.0:1788845036.858418:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641652864 00002000:00100000:1.0:1788845036.858469:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1114112->1118207) req@ffff94a03f6c6300 x1875739641652864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.858480:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c6300 with x1875739641652864 ext(1114112->1118207) 00002000:00010000:1.0:1788845036.858486:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c6300 00000100:00100000:1.0:1788845036.858514:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 430 00000100:00100000:1.0:1788845036.858523:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641652864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.858806:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641652864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 283us (696us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.858815:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 430 00002000:00100000:1.0:1788845036.858822:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1114112->1118207) req@ffff94a03f6c6300 x1875739641652864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.858831:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c6300 with x1875739641652864 ext(1114112->1118207) 00002000:00010000:1.0:1788845036.858834:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c6300 00000100:00100000:3.0:1788845036.862118:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.862227:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641652992 00000100:00100000:0.0:1788845036.862236:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641652992 00002000:00100000:0.0:1788845036.862306:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1118208->1122303) req@ffff94a03f6c5f80 x1875739641652992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.862318:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c5f80 with x1875739641652992 ext(1118208->1122303) 00002000:00010000:0.0:1788845036.862324:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c5f80 00000100:00100000:0.0:1788845036.862356:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 431 00000100:00100000:0.0:1788845036.862366:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641652992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.862611:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641652992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 245us (504us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.862619:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 431 00002000:00100000:0.0:1788845036.862626:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1118208->1122303) req@ffff94a03f6c5f80 x1875739641652992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.862634:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c5f80 with x1875739641652992 ext(1118208->1122303) 00002000:00010000:0.0:1788845036.862637:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c5f80 00000100:00100000:2.0:1788845036.867025:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.867138:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641653120 00000100:00100000:1.0:1788845036.867148:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641653120 00002000:00100000:1.0:1788845036.867202:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1122304->1126399) req@ffff949f02365f80 x1875739641653120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.867214:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365f80 with x1875739641653120 ext(1122304->1126399) 00002000:00010000:1.0:1788845036.867220:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365f80 00000100:00100000:1.0:1788845036.867255:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 432 00000100:00100000:1.0:1788845036.867265:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641653120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845036.867547:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641653120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 282us (532us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845036.867555:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 432 00002000:00100000:1.0:1788845036.867564:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1122304->1126399) req@ffff949f02365f80 x1875739641653120/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845036.867572:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365f80 with x1875739641653120 ext(1122304->1126399) 00002000:00010000:1.0:1788845036.867576:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365f80 00000100:00100000:2.0:1788845036.871103:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.871209:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641653248 00000100:00100000:0.0:1788845036.871220:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641653248 00000100:00100000:0.0:1788845036.871281:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1229 00000100:00100000:0.0:1788845036.871292:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641653248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.871322:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.871343:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adb62 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a925f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.871380:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.871423:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010482a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660314112:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.871508:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.873941:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010482a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660314112:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.873969:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:0.0:1788845036.873990:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adb62 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a925f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.874010:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.874013:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.874017:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adb62 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a925f expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.874033:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:0.0:1788845036.874093:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641653248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2802us (3000us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.874102:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1229 00000100:00100000:3.0:1788845036.877511:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.877605:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641653376 00000100:00100000:0.0:1788845036.877616:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641653376 00002000:00100000:0.0:1788845036.877673:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1126400->1130495) req@ffff94a03f6c4e00 x1875739641653376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/0 e 0 to 0 dl 1788845047 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.877685:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c4e00 with x1875739641653376 ext(1126400->1130495) 00002000:00010000:0.0:1788845036.877691:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c4e00 00000100:00100000:0.0:1788845036.877723:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 433 00000100:00100000:0.0:1788845036.877733:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641653376:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845036.878027:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641653376:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 295us (526us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845036.878034:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 433 00002000:00100000:0.0:1788845036.878041:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1126400->1130495) req@ffff94a03f6c4e00 x1875739641653376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:142/0 lens 488/4536 e 0 to 0 dl 1788845047 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845036.878049:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c4e00 with x1875739641653376 ext(1126400->1130495) 00002000:00010000:0.0:1788845036.878053:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c4e00 00000100:00100000:2.0:1788845036.883807:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.883930:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641653504 00000100:00100000:1.0:1788845036.883941:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641653504 00000100:00100000:1.0:1788845036.883994:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 846 00000100:00100000:1.0:1788845036.884004:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641653504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.884025:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.884044:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9266 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.884072:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:1.0:1788845036.884077:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:1.0:1788845036.884087:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.884106:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.884120:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845036.884189:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641653504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 186us (423us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.884196:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 846 00000100:00100000:2.0:1788845036.889444:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.889600:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641653632 00000100:00100000:1.0:1788845036.889608:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641653632 00000100:00100000:1.0:1788845036.889673:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1230 00000100:00100000:1.0:1788845036.889683:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641653632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.889705:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.889726:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adb70 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a926d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.889767:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.889817:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660314240:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.889937:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.892122:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660314240:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.892149:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:1.0:1788845036.892169:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adb70 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a926d expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.892190:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.892193:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.892199:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adb70 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a926d expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.892214:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845036.892277:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641653632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2594us (2844us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.892286:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1230 00000100:00100000:2.0:1788845036.895834:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.896060:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641653760 00000100:00100000:0.0:1788845036.896069:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641653760 00000100:00100000:0.0:1788845036.896133:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1231 00000100:00100000:0.0:1788845036.896144:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641653760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.896166:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.896186:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adb77 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9274 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.896222:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.896265:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010482a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660314368:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845036.896363:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.899161:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010482a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660314368:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.899188:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:0.0:1788845036.899207:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adb77 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9274 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.899233:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.899236:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.899240:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adb77 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9274 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.899256:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:0.0:1788845036.899330:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641653760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3188us (3508us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.899338:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1231 00000100:00100000:3.0:1788845036.903872:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.904125:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641653888 00000100:00100000:1.0:1788845036.904137:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641653888 00000100:00100000:1.0:1788845036.904201:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 847 00000100:00100000:1.0:1788845036.904212:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641653888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.904230:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.904250:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a927b expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.904284:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.904295:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.904304:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:1.0:1788845036.904309:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:1.0:1788845036.904315:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1052672->1110015] (req 1052672->1110015) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.904351:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.904372:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 10 pid: 6682 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.904387:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660314496:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.904643:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.904857:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 9 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.904874:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660314624:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845036.904932:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.911033:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.911075:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.911164:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641654144 00000100:00100000:0.0:1788845036.911174:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641654144 00010000:00010000:0.0:1788845036.911214:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 10 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.911252:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641654016 00000100:00100000:0.0:1788845036.911256:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641654016 00010000:00010000:0.0:1788845036.911269:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 11 pid: 6682 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.911294:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 423 00000100:00100000:0.0:1788845036.911304:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641654144:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845036.911318:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845036.911341:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 10 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.911358:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 10 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.911388:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845036.911445:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641654144:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 142us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.911452:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 423 00000100:00100000:0.0:1788845036.911481:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 424 00000100:00100000:0.0:1788845036.911488:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641654016:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845036.911496:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845036.911504:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 11 pid: 6682 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.911520:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 11 pid: 6682 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.911532:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:0.0:1788845036.911537:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:0.0:1788845036.911546:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.911557:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.911596:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845036.911610:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010482a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660314752:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845036.911657:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00010000:00010000:1.0:1788845036.911944:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb69 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9266 expref: 11 pid: 6682 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.911960:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660314496:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.911972:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd29ff85/0xb2187301287adb5b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1105920->1110015) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9258 expref: 9 pid: 7983 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.911982:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660314624:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.912010:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.912021:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845036.912048:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641653888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7838us (8187us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.912054:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 847 00000100:00100000:1.0:1788845036.914682:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010482a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660314752:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845036.914706:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845036.914756:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641654016:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3268us (3680us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.914767:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 424 00000100:00100000:0.0:1788845036.916467:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.916607:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641654272 00000100:00100000:1.0:1788845036.916616:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641654272 00000100:00100000:1.0:1788845036.916696:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 434 00000100:00100000:1.0:1788845036.916706:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641654272:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:1.0:1788845036.917029:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1127183 -> 1127183 00002000:00010000:1.0:1788845036.917033:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00002000:00010000:1.0:1788845036.917035:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845036 -> 1788845035 00002000:00010000:1.0:1788845036.917038:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00002000:00010000:1.0:1788845036.917040:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:1.0:1788845036.917100:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641654272:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 395us (649us total) trans 4294967445 rc 0/0 00000100:00100000:1.0:1788845036.917108:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 434 00000100:00100000:2.0:1788845036.921417:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.921629:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641654400 00000100:00100000:3.0:1788845036.921639:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641654400 00000100:00100000:3.0:1788845036.921712:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1232 00000100:00100000:3.0:1788845036.921722:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641654400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.921744:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.921766:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287adb85 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9282 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.921813:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.921854:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660314880:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.921967:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.925167:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660314880:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845036.925195:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:3.0:1788845036.925214:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287adb85 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9282 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.925236:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845036.925239:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845036.925244:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ee00/0xb2187301287adb85 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9282 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.925259:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ee00, rc 0) 00000100:00100000:3.0:1788845036.925330:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641654400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3609us (3925us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845036.925339:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1232 00000100:00100000:0.0:1788845036.928628:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.928851:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641654528 00000100:00100000:2.0:1788845036.928862:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641654528 00000100:00100000:2.0:1788845036.928956:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1233 00000100:00100000:2.0:1788845036.928967:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641654528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.928990:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.929013:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adb8c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9289 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.929050:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.929089:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660315008:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.929173:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.932132:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660315008:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845036.932163:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:0.0:1788845036.932183:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adb8c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9289 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.932205:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.932208:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845036.932214:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adb8c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9289 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.932230:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:0.0:1788845036.932297:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641654528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3332us (3680us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.932312:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1233 00000100:00100000:3.0:1788845036.935731:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.935905:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641654656 00000100:00100000:1.0:1788845036.935915:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641654656 00000100:00100000:1.0:1788845036.935964:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 848 00000100:00100000:1.0:1788845036.935974:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034509050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641654656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.935993:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.936011:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb93 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9290 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.936073:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660315136:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.936159:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845036.938860:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845035 -> 1788845036 00000100:00100000:1.0:1788845036.938923:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660315136:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.938958:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb93 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9290 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.938985:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287adb93 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9290 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.938996:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845036.939054:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034509050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641654656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3081us (3330us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.939062:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 848 00000100:00100000:2.0:1788845036.942751:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.945876:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641654784 00000100:00100000:3.0:1788845036.945935:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641654784 00000100:00100000:3.0:1788845036.946000:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 849 00000100:00100000:3.0:1788845036.946011:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641654784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.946033:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.946053:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8699c30/0xb2187301287adb9a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9297 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.946115:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660315264:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.946217:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845036.948970:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660315264:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845036.949005:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b8699c30/0xb2187301287adb9a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9297 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.949030:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8699c30/0xb2187301287adb9a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9297 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.949041:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e800, rc 0) 00000100:00100000:0.0:1788845036.949093:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641654784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3083us (6352us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845036.949103:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 849 00000100:00100000:1.0:1788845036.953824:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.953977:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641654912 00000100:00100000:2.0:1788845036.953988:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641654912 00000100:00100000:2.0:1788845036.954041:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1234 00000100:00100000:2.0:1788845036.954048:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641654912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.954065:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.954082:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adba1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a929e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.954114:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.954146:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660315392:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.954217:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.956102:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660315392:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845036.956129:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:1.0:1788845036.956146:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adba1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a929e expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.956167:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.956170:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845036.956174:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adba1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a929e expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.956189:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:1.0:1788845036.956244:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641654912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2196us (2429us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845036.956387:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1234 00000100:00100000:0.0:1788845036.958918:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845036.959046:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641655040 00000100:00100000:2.0:1788845036.959056:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641655040 00000100:00100000:2.0:1788845036.959105:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1235 00000100:00100000:2.0:1788845036.959113:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641655040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.959131:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.959147:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adba8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92a5 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.959175:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.959208:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660315520:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845036.959278:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845036.961173:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660315520:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845036.961193:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 453107 00010000:00010000:2.0:1788845036.961209:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adba8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92a5 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.961224:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845036.961226:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845036.961229:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adba8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92a5 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.961241:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:2.0:1788845036.961292:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641655040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2178us (2410us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845036.961305:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1235 00000100:00100000:3.0:1788845036.965744:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.966594:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641655296 00000100:00100000:1.0:1788845036.966605:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641655296 00000100:00100000:1.0:1788845036.966678:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1236 00000100:00100000:1.0:1788845036.966688:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641655296:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845036.966743:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845036.966770:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adae4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91e8 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.966801:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.966806:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845036.967034:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.967050:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.967059:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.967093:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.967113:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 12 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.967127:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660315648:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845036.967223:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.970293:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.970474:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660315648:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845036.970503:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.970609:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641655424 00000100:00100000:1.0:1788845036.970616:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641655424 00010000:00010000:1.0:1788845036.970647:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.970690:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 425 00000100:00100000:1.0:1788845036.970699:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d48050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641655424:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845036.970713:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845036.970719:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.970734:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.970754:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14800/0xb2187301287adb46 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9243 expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.970768:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.970772:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.970793:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.970800:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.970826:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845036.970874:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d48050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641655424:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 175us (589us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.970908:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 425 00010000:00010000:3.0:1788845036.971044:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845036.971053:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845036.971162:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967742 00000004:00010000:3.0:1788845036.971206:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 453107 -> 453107 00000004:00010000:3.0:1788845036.971210:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:3.0:1788845036.971213:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:3.0:1788845036.971215:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:3.0:1788845036.971217:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000004:00010000:3.0:1788845036.971224:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a728787d reply state 00000000948bb955 transno 4294967743 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.971233:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845036.971461:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641655296:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4775us (5822us total) trans 4294967743 rc 0/0 00000100:00100000:3.0:1788845036.971469:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1236 00000020:00080000:3.0:1788845036.971473:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845036.974055:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:1.0:1788845036.974299:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.974314:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.974323:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.974329:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d15c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845036.974335:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.974344:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.974365:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.974368:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845036.974373:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.974379:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbaf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.975160:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641655552 00000100:00100000:2.0:1788845036.975173:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641655552 00000100:00100000:2.0:1788845036.975282:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1237 00000100:00100000:2.0:1788845036.975302:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641655552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845036.975329:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845036.975349:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92ac expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.975379:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.975390:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.975423:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.975443:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 11 pid: 5840 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.975456:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660315776:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845036.975544:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845036.980001:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.980260:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641655680 00000100:00100000:1.0:1788845036.980270:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641655680 00010000:00010000:1.0:1788845036.980322:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.980364:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 426 00000100:00100000:1.0:1788845036.980374:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641655680:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845036.980391:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845036.980416:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.980433:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.980455:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.980462:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.980473:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.980482:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.980518:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.980532:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660315904:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845036.980605:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00010000:00010000:2.0:1788845036.981746:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287adac1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a91da expref: 12 pid: 5840 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845036.981772:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660315776:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845036.981808:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845036.981822:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845036.981825:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845036.981828:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:2.0:1788845036.981879:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641655552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6588us (7833us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845036.981913:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1237 00000100:00100000:3.0:1788845036.984953:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.984979:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.985061:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641655808 00000100:00100000:1.0:1788845036.985069:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641655808 00002000:00010000:1.0:1788845036.985106:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845036.985110:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c4380 with x1875739641655808 00000100:00100000:1.0:1788845036.985133:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 833 00000100:00100000:1.0:1788845036.985142:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641655808:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845036.985328:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 453107 -> 578129 00000004:00010000:1.0:1788845036.985331:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.985334:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.985336:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845036.985338:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:1.0:1788845036.985396:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641655808:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 254us (415us total) trans 4294967744 rc 0/0 00000100:00100000:1.0:1788845036.985402:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 833 00002000:00010000:1.0:1788845036.985405:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845036.985408:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c4380 with x1875739641655808 00000100:00100000:1.0:1788845036.985449:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660315904:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845036.985467:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845036.985485:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641655680:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5113us (5496us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.985493:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 426 00000100:00100000:0.0:1788845036.985702:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641655936 00000100:00100000:0.0:1788845036.985706:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641655936 00002000:00010000:0.0:1788845036.985725:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845036.985729:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c7100 with x1875739641655936 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845036.985736:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c7100. 00000100:00100000:0.0:1788845036.985763:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 435 00000100:00100000:0.0:1788845036.985770:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641655936:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845036.985980:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1127183 -> 0 00002000:00010000:0.0:1788845036.985983:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845036 -> 1788845036 00002000:00010000:0.0:1788845036.985986:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845036 -> 1788845035 00002000:00010000:0.0:1788845036.985988:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845036 -> 1788845036 00002000:00010000:0.0:1788845036.985991:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:0.0:1788845036.986015:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641655936:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 245us (1069us total) trans 4294967446 rc 0/0 00000100:00100000:0.0:1788845036.986021:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 435 00002000:00010000:0.0:1788845036.986024:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845036.986027:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c7100 with x1875739641655936 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845036.986031:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c7100. 00000100:00100000:3.0:1788845036.988915:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845036.989033:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641656064 00000100:00100000:1.0:1788845036.989044:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641656064 00000100:00100000:1.0:1788845036.989106:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1238 00000100:00100000:1.0:1788845036.989118:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641656064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.989143:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.989164:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287adbbd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92b3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.989249:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adbc4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.989265:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adbc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.989279:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adbc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845036.989319:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 578129 00000004:00200000:1.0:1788845036.989323:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845036.989342:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adbc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.989368:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287adbbd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92b3 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.989386:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adbc4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92b3 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.989394:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845036.989397:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845036.989401:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845036.989451:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641656064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 336us (547us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845036.989458:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1238 00000100:00100000:3.0:1788845036.993007:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845036.993497:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641656192 00000100:00100000:0.0:1788845036.993507:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641656192 00000100:00100000:0.0:1788845036.993600:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1239 00000100:00100000:0.0:1788845036.993611:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641656192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845036.993633:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845036.993663:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287adbcb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92ba expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.993749:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.993764:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.993786:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845036.993816:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 578129 00000004:00200000:0.0:1788845036.993820:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845036.993838:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.993865:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287adbcb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.993913:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845036.993927:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845036.993929:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845036.993933:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031a00, rc 0) 00000100:00100000:0.0:1788845036.993993:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641656192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 384us (996us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845036.993999:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1239 00000100:00100000:3.0:1788845036.996468:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.997005:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845036.997033:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641656448 00000100:00100000:3.0:1788845036.997041:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641656448 00000100:00100000:3.0:1788845036.997093:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 850 00000100:00100000:3.0:1788845036.997103:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641656448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845036.997120:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845036.997136:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287adbd9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92c8 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845036.997191:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660316032:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845036.997276:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845036.997518:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641656320 00000100:00100000:1.0:1788845036.997524:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641656320 00000100:00100000:1.0:1788845036.997580:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1240 00000100:00100000:1.0:1788845036.997587:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641656320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845036.997598:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845036.997612:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbe0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92c1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.997640:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845036.997677:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660316160:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845036.997706:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00002000:00010000:1.0:1788845036.999913:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845035 -> 1788845036 00000100:00100000:1.0:1788845036.999942:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660316032:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845036.999976:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287adbd9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92c8 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845036.999999:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000092cfd1ea/0xb2187301287adbd9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92c8 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.000012:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191e400, rc 0) 00000100:00100000:1.0:1788845037.000060:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641656448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2958us (3057us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.000069:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 850 00000100:00100000:1.0:1788845037.000130:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660316160:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845037.000145:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:1.0:1788845037.000157:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbe0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92c1 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.000170:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.000173:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845037.000177:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287adbe0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92c1 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.000188:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845037.000210:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641656320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2623us (3751us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.000216:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1240 00000100:00100000:3.0:1788845037.002941:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.002980:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.003174:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641656704 00000100:00100000:1.0:1788845037.003183:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641656704 00000100:00100000:1.0:1788845037.003254:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 851 00000100:00100000:1.0:1788845037.003265:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641656704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.003283:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.003311:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adbe7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92d6 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.003368:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660316288:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.003439:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845037.003680:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641656576 00000100:00100000:0.0:1788845037.003686:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641656576 00000100:00100000:0.0:1788845037.003718:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1241 00000100:00100000:0.0:1788845037.003727:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641656576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.003738:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.003753:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287adbee lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92cf expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.003793:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.003817:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010480000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660316416:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845037.003845:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00080000:0.0:1788845037.004811:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1788845037.004818:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845037.008591:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660316288:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845037.008634:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adbe7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92d6 expref: 11 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.008767:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adbe7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92d6 expref: 11 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.008791:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:0.0:1788845037.008929:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641656704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5665us (5996us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845037.008938:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 851 00000100:00100000:0.0:1788845037.010198:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010480000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660316416:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845037.010219:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:0.0:1788845037.010234:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287adbee lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92cf expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.010251:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.010254:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845037.010258:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287adbee lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92cf expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.010271:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031400, rc 0) 00000100:00100000:0.0:1788845037.010350:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641656576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6624us (7368us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845037.010359:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1241 00000100:00100000:3.0:1788845037.020998:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.021546:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641656832 00000100:00100000:1.0:1788845037.021556:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641656832 00002000:00100000:1.0:1788845037.021612:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8ce00 x1875739641656832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.021621:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8ce00 with x1875739641656832 00000100:00100000:1.0:1788845037.021653:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 834 00000100:00100000:1.0:1788845037.021663:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641656832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00080000:0.0:1788845037.022096:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845037.031097:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641656832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 9421us (10094us total) trans 0 rc 208896/208896 00000100:00100000:1.0:1788845037.031115:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 834 00002000:00100000:1.0:1788845037.031130:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8ce00 x1875739641656832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 208896/208896 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.031141:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8ce00 with x1875739641656832 00000100:00100000:1.0:1788845037.035587:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.035836:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641656960 00000100:00100000:2.0:1788845037.035847:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641656960 00000100:00100000:2.0:1788845037.035975:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 852 00000100:00100000:2.0:1788845037.035987:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641656960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.036007:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.036025:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92dd expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.036056:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.036064:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:2.0:1788845037.036068:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 8) 00010000:00010000:2.0:1788845037.036073:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1699840->1703935] (req 1699840->1703935) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.036111:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.036129:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 9 pid: 6685 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.036142:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660316544:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845037.036222:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.039453:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660316544:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845037.039505:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1699840->1703935] (req 1699840->1703935) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.039526:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032800, rc 0) 00000100:00100000:2.0:1788845037.039593:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641656960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3608us (4015us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.039602:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 852 00000100:00100000:3.0:1788845037.040499:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.040629:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641657088 00000100:00100000:1.0:1788845037.040638:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641657088 00010000:00010000:1.0:1788845037.040961:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 10 pid: 6685 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.041001:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 427 00000100:00100000:1.0:1788845037.041010:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641657088:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845037.041024:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845037.041049:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 10 pid: 6685 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.041067:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 10 pid: 6685 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.041093:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adb7e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1052672->1110015) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a927b expref: 10 pid: 6685 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.041117:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 8) 00010000:00010000:1.0:1788845037.041121:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 8) 00010000:00010000:1.0:1788845037.041130:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.041141:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.041179:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.041193:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660316672:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845037.041267:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00080000:0.0:1788845037.042803:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845037.045408:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660316672:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845037.045438:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845037.045593:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641657088:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4582us (5097us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.045604:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 427 00000100:00100000:3.0:1788845037.048870:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.049077:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641657216 00000100:00100000:1.0:1788845037.049088:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641657216 00000100:00100000:1.0:1788845037.049188:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1242 00000100:00100000:1.0:1788845037.049200:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641657216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.049221:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.049242:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adbfc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92e4 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.049281:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.049407:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660316800:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.049513:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788845037.053052:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845037 00000100:00100000:1.0:1788845037.053085:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660316800:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845037.053108:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:1.0:1788845037.053129:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adbfc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92e4 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.053148:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.053151:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845037.053157:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adbfc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92e4 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.053172:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845037.053236:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641657216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4038us (4375us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.053245:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1242 00000100:00100000:3.0:1788845037.056738:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.056843:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641657344 00000100:00100000:2.0:1788845037.056853:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641657344 00000100:00100000:2.0:1788845037.056946:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1243 00000100:00100000:2.0:1788845037.056956:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641657344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.056977:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.056998:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032600/0xb2187301287adc03 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92eb expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.057040:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.057078:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660316928:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.057159:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.059998:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660316928:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845037.060021:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:2.0:1788845037.060038:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032600/0xb2187301287adc03 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92eb expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.060057:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.060060:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845037.060064:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032600/0xb2187301287adc03 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92eb expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.060078:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032600, rc 0) 00000100:00100000:2.0:1788845037.060160:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641657344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3205us (3431us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.060169:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1243 00000100:00100000:3.0:1788845037.065738:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.066068:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641657472 00000100:00100000:0.0:1788845037.066081:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641657472 00000100:00100000:0.0:1788845037.066147:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1244 00000100:00100000:0.0:1788845037.066159:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641657472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.066185:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.066208:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287adc0a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92f2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.066249:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.066294:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010480000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660317056:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845037.066472:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.069624:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010480000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660317056:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845037.069720:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:2.0:1788845037.069742:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287adc0a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92f2 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.069764:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.069768:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845037.069783:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287adc0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92f2 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.069799:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031400, rc 0) 00000100:00100000:2.0:1788845037.069977:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641657472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3820us (4248us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.069986:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1244 00000100:00100000:3.0:1788845037.073342:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.073436:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641657600 00000100:00100000:0.0:1788845037.073446:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641657600 00000100:00100000:0.0:1788845037.073521:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1245 00000100:00100000:0.0:1788845037.073532:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641657600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.073555:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.073578:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287adc11 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a92f9 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.073616:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.073681:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010483480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660317184:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845037.073782:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845037.076684:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010483480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660317184:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845037.076714:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:0.0:1788845037.076735:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287adc11 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92f9 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.076757:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.076760:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845037.076766:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287adc11 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92f9 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.076795:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:0.0:1788845037.076859:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641657600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3328us (3526us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845037.076867:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1245 00000100:00100000:3.0:1788845037.084629:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.084748:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641657728 00000100:00100000:1.0:1788845037.084759:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641657728 00002000:00100000:1.0:1788845037.084839:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b100 x1875739641657728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.084851:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b100 with x1875739641657728 00000100:00100000:1.0:1788845037.084929:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 835 00000100:00100000:1.0:1788845037.084939:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641657728:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845037.093239:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641657728:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8290us (8609us total) trans 0 rc 372736/372736 00000100:00100000:1.0:1788845037.093253:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 835 00002000:00100000:1.0:1788845037.093268:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b100 x1875739641657728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 372736/372736 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.093278:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b100 with x1875739641657728 00000100:00100000:2.0:1788845037.099182:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.099296:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641657856 00000100:00100000:1.0:1788845037.099365:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641657856 00000100:00100000:1.0:1788845037.099479:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 853 00000100:00100000:1.0:1788845037.099489:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641657856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.099508:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.099526:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287adc18 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9300 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.099581:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660317312:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.099798:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845037.106174:0:7983:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1703240 00002000:00010000:1.0:1788845037.106183:0:7983:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845036 -> 1788845037 00002000:00010000:1.0:1788845037.106185:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845036 -> 1788845037 00002000:00010000:1.0:1788845037.106187:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845036 -> 1788845037 00000100:00100000:1.0:1788845037.106212:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660317312:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.106248:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287adc18 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9300 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.106274:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af064cb5/0xb2187301287adc18 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9300 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.106289:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845037.106536:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641657856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7047us (7362us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.106546:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 853 00000100:00100000:3.0:1788845037.109734:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.109879:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641657984 00000100:00100000:1.0:1788845037.109908:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641657984 00000100:00100000:1.0:1788845037.109970:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 854 00000100:00100000:1.0:1788845037.109980:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641657984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.109999:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.110016:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adc1f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9307 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.110077:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660317440:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.110161:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.112507:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660317440:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.112548:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adc1f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9307 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.112575:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adc1f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9307 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.112588:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845037.112647:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641657984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2667us (2922us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.112663:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 854 00000100:00100000:0.0:1788845037.116615:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.120531:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641658112 00000100:00100000:1.0:1788845037.120547:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641658112 00000100:00100000:1.0:1788845037.120942:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1246 00000100:00100000:1.0:1788845037.120963:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641658112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.120985:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.121006:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adc26 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a930e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.121043:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.121085:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660317568:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.121193:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4c0c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788845037.128667:0:5839:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845036 -> 1788845037 00000004:00010000:1.0:1788845037.128678:0:5839:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845036 -> 1788845037 00000100:00100000:1.0:1788845037.128705:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660317568:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845037.128731:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:1.0:1788845037.128751:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adc26 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a930e expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.128785:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.128789:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845037.128794:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adc26 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a930e expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.128812:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845037.128934:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641658112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7932us (12282us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.129382:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1246 00000100:00100000:0.0:1788845037.133054:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.133291:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641658240 00000100:00100000:3.0:1788845037.133311:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641658240 00000100:00100000:3.0:1788845037.133405:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1247 00000100:00100000:3.0:1788845037.133416:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641658240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.133447:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.133468:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287adc2d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9315 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.133507:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.133546:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660317696:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845037.133629:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.136791:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660317696:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845037.136821:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:3.0:1788845037.136845:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287adc2d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9315 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.136869:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845037.136872:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845037.136878:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287adc2d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9315 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.141900:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845037.145065:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845037.145215:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641658240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11796us (12263us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845037.145225:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1247 00000100:00100000:2.0:1788845037.152333:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.152451:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641658368 00000100:00100000:0.0:1788845037.152462:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641658368 00000100:00100000:0.0:1788845037.152523:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1248 00000100:00100000:0.0:1788845037.152534:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641658368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.152556:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.152577:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287adc34 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a931c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.152617:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.152670:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010480e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660317824:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845037.152760:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.166053:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010480e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660317824:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845037.166080:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:2.0:1788845037.166103:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287adc34 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a931c expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.166127:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.166131:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845037.166137:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287adc34 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a931c expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.166153:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:2.0:1788845037.166232:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641658368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13699us (13906us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.166242:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1248 00000100:00100000:2.0:1788845037.170804:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.170984:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641658496 00000100:00100000:0.0:1788845037.170994:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641658496 00000100:00100000:0.0:1788845037.171053:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1249 00000100:00100000:0.0:1788845037.171063:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641658496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.171085:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.171107:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9323 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.171135:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.171147:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.171182:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.171212:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 17 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.171227:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010482d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660317952:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845037.171565:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845037.183550:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010482d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660317952:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845037.183589:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.183609:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.183612:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845037.183617:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.183629:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:0.0:1788845037.183770:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641658496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12707us (12974us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.183787:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1249 00000100:00100000:1.0:1788845037.183919:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.184094:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641658624 00000100:00100000:3.0:1788845037.184102:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641658624 00010000:00010000:3.0:1788845037.184139:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 18 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.184170:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 428 00000100:00100000:3.0:1788845037.184179:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641658624:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845037.184192:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845037.184218:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 18 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.184236:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 18 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.184259:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287adbb6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ac expref: 18 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.184273:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845037.184279:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.187022:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.187053:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.187115:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.187134:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660318080:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845037.187217:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.195597:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660318080:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845037.195622:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845037.195689:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641658624:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11510us (11769us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845037.195700:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 428 00000100:00100000:1.0:1788845037.205179:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.205233:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.205502:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641658880 00000100:00100000:0.0:1788845037.205512:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641658880 00002000:00100000:0.0:1788845037.205560:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1130496->1699839) req@ffff94a0346faa00 x1875739641658880/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.205571:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0346faa00 with x1875739641658880 ext(1130496->1699839) 00002000:00010000:0.0:1788845037.205577:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0346faa00 00000100:00100000:0.0:1788845037.205611:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 436 00000100:00100000:0.0:1788845037.205621:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641658880:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.206417:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641658752 00000100:00100000:2.0:1788845037.206427:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641658752 00002000:00100000:2.0:1788845037.206464:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8000 x1875739641658752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.206475:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8000 with x1875739641658752 00000100:00100000:2.0:1788845037.206508:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 836 00000100:00100000:2.0:1788845037.206516:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641658752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.207229:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641658752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 710us (1992us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.207238:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 836 00002000:00100000:2.0:1788845037.207246:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8000 x1875739641658752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.207266:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8000 with x1875739641658752 00000100:00100000:0.0:1788845037.214059:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.219660:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641658880:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14018us (14468us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.219677:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 436 00002000:00100000:1.0:1788845037.219692:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1130496->1699839) req@ffff94a0346faa00 x1875739641658880/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.219703:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0346faa00 with x1875739641658880 ext(1130496->1699839) 00002000:00010000:1.0:1788845037.219710:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0346faa00 00000100:00100000:2.0:1788845037.220278:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641659008 00000100:00100000:2.0:1788845037.220289:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641659008 00002000:00100000:2.0:1788845037.220345:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482d80 x1875739641659008/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.220356:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482d80 with x1875739641659008 00000100:00100000:2.0:1788845037.220816:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 837 00000100:00100000:2.0:1788845037.220828:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641659008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.221094:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641659008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (7043us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.221102:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 837 00002000:00100000:2.0:1788845037.221110:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482d80 x1875739641659008/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.221118:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482d80 with x1875739641659008 00000100:00100000:1.0:1788845037.235116:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.235202:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641659136 00000100:00100000:2.0:1788845037.235214:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641659136 00002000:00100000:2.0:1788845037.235264:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9880 x1875739641659136/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.235275:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9880 with x1875739641659136 00000100:00100000:2.0:1788845037.235493:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 838 00000100:00100000:2.0:1788845037.235505:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641659136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.235864:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641659136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 360us (757us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.235873:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 838 00002000:00100000:2.0:1788845037.235881:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9880 x1875739641659136/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.235923:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9880 with x1875739641659136 00000100:00100000:3.0:1788845037.240239:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.241351:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641659264 00000100:00100000:2.0:1788845037.241362:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641659264 00002000:00100000:0.0:1788845037.244351:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f99f80 x1875739641659264/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.244374:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f99f80 with x1875739641659264 00000100:00100000:0.0:1788845037.244415:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 839 00000100:00100000:0.0:1788845037.244426:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641659264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845037.245357:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641659264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 927us (5124us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.245368:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 839 00002000:00100000:0.0:1788845037.245377:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f99f80 x1875739641659264/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.245387:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f99f80 with x1875739641659264 00000100:00100000:0.0:1788845037.251703:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.251814:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641659392 00000100:00100000:1.0:1788845037.251825:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641659392 00002000:00100000:1.0:1788845037.251876:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482680 x1875739641659392/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.251932:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482680 with x1875739641659392 00000100:00100000:1.0:1788845037.251969:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 840 00000100:00100000:1.0:1788845037.251979:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641659392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845037.252513:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641659392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 532us (817us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.252522:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 840 00002000:00100000:1.0:1788845037.252530:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482680 x1875739641659392/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.252544:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482680 with x1875739641659392 00000100:00100000:1.0:1788845037.262117:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.266602:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660318208:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845037.266688:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845037.266920:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660318208 00000100:00100000:1.0:1788845037.266929:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660318208 00000100:00100000:1.0:1788845037.266998:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 82 00000100:00100000:1.0:1788845037.267007:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660318208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788845037.267044:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788845037.267094:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660318208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (406us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.267101:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 82 00000100:00100000:1.0:1788845037.267148:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660318208:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1788845037.267213:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845037.267352:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641659520 00000100:00100000:2.0:1788845037.267357:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641659520 00002000:00100000:2.0:1788845037.267394:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa680 x1875739641659520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.267404:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa680 with x1875739641659520 00000100:00100000:2.0:1788845037.267422:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 841 00000100:00100000:2.0:1788845037.267430:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641659520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.267678:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641659520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (5572us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.267686:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 841 00002000:00100000:2.0:1788845037.267692:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa680 x1875739641659520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.267701:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa680 with x1875739641659520 00000100:00100000:3.0:1788845037.271475:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.271649:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641659648 00000100:00100000:1.0:1788845037.271659:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641659648 00002000:00100000:1.0:1788845037.271710:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b480 x1875739641659648/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.271721:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b480 with x1875739641659648 00000100:00100000:1.0:1788845037.271763:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 842 00000100:00100000:1.0:1788845037.271782:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641659648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845037.272077:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641659648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 304us (609us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.272085:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 842 00002000:00100000:1.0:1788845037.272093:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b480 x1875739641659648/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.272099:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b480 with x1875739641659648 00000100:00100000:1.0:1788845037.280624:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.282418:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641659776 00000100:00100000:2.0:1788845037.282432:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641659776 00002000:00100000:2.0:1788845037.282488:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9c00 x1875739641659776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.282500:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9c00 with x1875739641659776 00000100:00100000:0.0:1788845037.282627:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 843 00000100:00100000:0.0:1788845037.282639:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641659776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845037.282933:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641659776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 296us (2320us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.282942:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 843 00002000:00100000:0.0:1788845037.282949:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9c00 x1875739641659776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.282957:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9c00 with x1875739641659776 00000100:00100000:0.0:1788845037.287406:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.287860:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641659904 00000100:00100000:1.0:1788845037.287872:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641659904 00002000:00100000:1.0:1788845037.288153:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480000 x1875739641659904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.288165:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480000 with x1875739641659904 00000100:00100000:1.0:1788845037.288311:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 844 00000100:00100000:1.0:1788845037.288321:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641659904:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845037.288598:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641659904:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 278us (1200us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.288606:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 844 00002000:00100000:1.0:1788845037.288614:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010480000 x1875739641659904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.288621:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010480000 with x1875739641659904 00000100:00100000:3.0:1788845037.292035:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.292252:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641660032 00000100:00100000:0.0:1788845037.292263:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641660032 00002000:00100000:0.0:1788845037.292319:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f98000 x1875739641660032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.292330:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98000 with x1875739641660032 00000100:00100000:0.0:1788845037.292367:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 845 00000100:00100000:0.0:1788845037.292377:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641660032:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845037.292643:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641660032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 267us (616us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.292850:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 845 00002000:00100000:0.0:1788845037.292858:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f98000 x1875739641660032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.292866:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98000 with x1875739641660032 00000100:00100000:0.0:1788845037.303795:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.304011:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641660160 00000100:00100000:2.0:1788845037.304022:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641660160 00002000:00100000:0.0:1788845037.304204:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482a00 x1875739641660160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.304215:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482a00 with x1875739641660160 00000100:00100000:0.0:1788845037.304312:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 846 00000100:00100000:0.0:1788845037.304322:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641660160:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845037.304568:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641660160:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 246us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.304576:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 846 00002000:00100000:0.0:1788845037.304583:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482a00 x1875739641660160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.304591:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482a00 with x1875739641660160 00000100:00100000:3.0:1788845037.308746:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.308793:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.308847:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641660288 00000100:00100000:0.0:1788845037.308857:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641660288 00000100:00100000:0.0:1788845037.308908:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1250 00000100:00100000:0.0:1788845037.308915:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641660288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.308932:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.308948:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287adc42 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a932a expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.308982:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287adc42 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a932a expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845037.308995:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 578129 00010000:00010000:0.0:1788845037.309002:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287adc42 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a932a expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.309016:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.309018:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845037.309021:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:2.0:1788845037.309110:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641660416 00000100:00100000:2.0:1788845037.309114:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641660416 00000100:00100000:2.0:1788845037.309148:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 855 00000100:00100000:2.0:1788845037.309153:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641660416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.309160:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.309168:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287adc49 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9331 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.309215:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660318336:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845037.310425:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641660288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1504us (1681us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.310435:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1250 00000100:00100000:2.0:1788845037.310535:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.316851:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660318336:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845037.316917:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287adc49 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9331 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.316935:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287adc49 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9331 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.316951:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:2.0:1788845037.317002:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641660416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7846us (8205us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.317010:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 855 00000100:00100000:1.0:1788845037.322613:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.322786:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641660544 00000100:00100000:2.0:1788845037.322796:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641660544 00000100:00100000:2.0:1788845037.322912:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 856 00000100:00100000:2.0:1788845037.322924:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641660544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.322944:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.322963:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287adc50 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9338 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.323021:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660318464:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.323114:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.328873:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660318464:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.328939:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287adc50 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9338 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.328965:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287adc50 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9338 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.328979:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:1.0:1788845037.329044:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641660544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6122us (6441us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.329055:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 856 00000100:00100000:3.0:1788845037.334746:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.334846:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641660800 00000100:00100000:1.0:1788845037.334856:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641660800 00000100:00100000:1.0:1788845037.334948:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 39 00000100:00100000:1.0:1788845037.334959:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00777d050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641660800:12345-192.168.201.29@tcp:35:fsx.0 00000001:00080000:1.0:1788845037.335209:0:5842:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967745 00000100:00100000:1.0:1788845037.335464:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00777d050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641660800:12345-192.168.201.29@tcp:35:fsx.0 Request processed in 505us (725us total) trans 4294967746 rc 0/0 00000100:00100000:1.0:1788845037.335472:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 39 00000020:00080000:1.0:1788845037.335477:0:5842:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845037.336829:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.336872:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.337003:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641660928 00000100:00100000:3.0:1788845037.337012:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641660928 00000100:00100000:1.0:1788845037.337114:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641661056 00000100:00100000:1.0:1788845037.337119:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641661056 00000100:00100000:1.0:1788845037.337157:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 123 00000100:00100000:1.0:1788845037.337167:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641661056:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845037.337234:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641661056:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 68us (411us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.337242:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 123 00000100:00100000:3.0:1788845037.338913:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 283 00000100:00100000:3.0:1788845037.338933:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2370:x1875739641660928:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845037.338995:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2370:x1875739641660928:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 71us (2122us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845037.339003:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 283 00000100:00100000:0.0:1788845037.344612:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.344759:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641661312 00000100:00100000:3.0:1788845037.344786:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641661312 00000100:00100000:3.0:1788845037.344847:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1251 00000100:00100000:3.0:1788845037.344859:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641661312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.345628:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.345789:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287adc57 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a933f expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.345868:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.345899:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.345912:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.345965:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.345974:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.345982:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347162:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287adc6c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347181:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287adc6c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347187:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287adc6c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347194:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287adc6c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347201:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191fe00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845037.347205:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287adc6c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347213:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287adc6c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347225:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845037.347230:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287adc6c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347237:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287adc6c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845037.347254:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1703240 00010000:00010000:3.0:1788845037.347799:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347807:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347812:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347818:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191f600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845037.347821:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347827:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347840:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:3.0:1788845037.347843:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845037.347846:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347852:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287adc5e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347861:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347867:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347872:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.347877:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f0191ea00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845037.347879:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.348078:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.348087:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3d6fced7 (ffff94a00be8d400) 00010000:00010000:3.0:1788845037.348089:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845037.348092:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.348098:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191ea00/0xb2187301287adc65 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.348151:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287adc57 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a933f expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.348161:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:3.0:1788845037.348163:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788845037.348166:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287adc57 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a933f expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.348176:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845037.348245:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641661312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3387us (3642us total) trans 4294967747 rc 301/301 00000100:00100000:3.0:1788845037.348254:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1251 00000100:00100000:2.0:1788845037.359631:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.361601:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641661440 00000100:00100000:1.0:1788845037.361617:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641661440 00000100:00100000:1.0:1788845037.361793:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 857 00000100:00100000:1.0:1788845037.361806:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641661440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.361827:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.361846:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287adc7a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9346 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.361938:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660318592:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.362020:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.365168:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660318592:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.365208:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287adc7a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9346 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.365234:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287adc7a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9346 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.365248:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845037.365325:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641661440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3521us (5704us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.365334:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 857 00000100:00100000:0.0:1788845037.368834:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.368932:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641661568 00000100:00100000:1.0:1788845037.368940:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641661568 00000100:00100000:1.0:1788845037.369001:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 858 00000100:00100000:1.0:1788845037.369011:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641661568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.369028:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.369046:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adc81 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a934d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.369102:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660318720:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.369168:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.372711:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660318720:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.372754:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adc81 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a934d expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.372794:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adc81 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a934d expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.372808:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845037.372859:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641661568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3849us (4036us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.372867:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 858 00000100:00100000:0.0:1788845037.383709:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.383809:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641661696 00000100:00100000:2.0:1788845037.383819:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641661696 00000100:00100000:2.0:1788845037.383878:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 859 00000100:00100000:2.0:1788845037.383923:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641661696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.383941:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.383959:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adc88 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9354 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.384017:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660318848:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.384102:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.391957:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660318848:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845037.391996:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adc88 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9354 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.392022:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adc88 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9354 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.392037:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033800, rc 0) 00000100:00100000:2.0:1788845037.392094:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641661696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8173us (8395us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.392103:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 859 00000100:00100000:2.0:1788845037.402846:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.402934:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641661824 00000100:00100000:0.0:1788845037.402945:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641661824 00000100:00100000:2.0:1788845037.403046:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1252 00000100:00100000:2.0:1788845037.403059:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641661824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.403089:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.403093:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845037.403142:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287adc42 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a932a expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.403223:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.403228:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845037.403243:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a935b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.406031:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.406045:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.406082:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.406099:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 12 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.406117:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660318976:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845037.406215:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.412836:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660318976:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845037.412917:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.412938:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.412941:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845037.412946:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.412958:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845037.413017:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641661824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9961us (10181us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.413026:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1252 00000100:00100000:3.0:1788845037.413475:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.413946:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641661952 00000100:00100000:2.0:1788845037.413954:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641661952 00010000:00010000:2.0:1788845037.413989:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 13 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.414021:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 429 00000100:00100000:2.0:1788845037.414030:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641661952:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845037.414043:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845037.414067:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 13 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.414083:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 13 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.414106:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287adc3b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9323 expref: 13 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.414121:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.414127:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.414137:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.414146:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.414178:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.414200:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660319104:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845037.414268:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.416593:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660319104:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845037.416619:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845037.416672:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641661952:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2643us (3198us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.416683:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 429 00000100:00100000:3.0:1788845037.417447:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.417781:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641662080 00000100:00100000:2.0:1788845037.417789:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641662080 00000100:00100000:2.0:1788845037.417835:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 860 00000100:00100000:2.0:1788845037.417844:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641662080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.417862:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.417993:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adc96 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9362 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.418049:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660319232:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.418218:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.428591:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660319232:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845037.428633:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adc96 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9362 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.428663:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adc96 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9362 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.428677:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033800, rc 0) 00000100:00100000:2.0:1788845037.428742:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641662080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10897us (11297us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.428751:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 860 00000100:00100000:2.0:1788845037.438254:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.438413:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641662208 00000100:00100000:0.0:1788845037.438422:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641662208 00002000:00100000:0.0:1788845037.438470:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364a80 x1875739641662208/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.438482:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364a80 with x1875739641662208 00000100:00100000:0.0:1788845037.438526:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 847 00000100:00100000:0.0:1788845037.438536:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641662208:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845037.438822:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641662208:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 286us (576us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.438829:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 847 00002000:00100000:0.0:1788845037.438837:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02364a80 x1875739641662208/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.438845:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02364a80 with x1875739641662208 00000100:00100000:3.0:1788845037.444119:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.446788:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641662336 00000100:00100000:1.0:1788845037.446805:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641662336 00000100:00100000:1.0:1788845037.446965:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 861 00000100:00100000:1.0:1788845037.446978:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641662336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.446997:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.447017:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287adc9d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9369 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.447081:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660319360:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.447197:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.450534:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660319360:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845037.450569:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287adc9d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9369 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.450588:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287adc9d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9369 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.450598:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:3.0:1788845037.450657:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641662336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3683us (6554us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845037.450666:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 861 00000100:00100000:3.0:1788845037.455935:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.456033:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641662464 00000100:00100000:1.0:1788845037.456042:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641662464 00000100:00100000:1.0:1788845037.456091:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 862 00000100:00100000:1.0:1788845037.456099:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641662464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.456114:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.456130:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287adca4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9370 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.456189:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660319488:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.456270:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.460215:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660319488:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845037.460578:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287adca4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9370 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.460602:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287adca4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9370 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.460614:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:2.0:1788845037.460791:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641662464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4690us (4862us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.460800:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 862 00000100:00100000:0.0:1788845037.464992:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.465132:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641662592 00000100:00100000:1.0:1788845037.465142:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641662592 00002000:00100000:1.0:1788845037.465199:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482680 x1875739641662592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.465210:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482680 with x1875739641662592 00000100:00100000:1.0:1788845037.465249:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 848 00000100:00100000:1.0:1788845037.465269:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641662592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845037.465870:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641662592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 601us (885us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.465879:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 848 00002000:00100000:1.0:1788845037.465914:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482680 x1875739641662592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.465922:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482680 with x1875739641662592 00000100:00100000:2.0:1788845037.468615:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.468776:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641662720 00000100:00100000:1.0:1788845037.468785:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641662720 00000100:00100000:1.0:1788845037.468847:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 863 00000100:00100000:1.0:1788845037.468857:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641662720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.468875:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.468929:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adcab lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9377 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.468993:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660319616:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.469067:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.471851:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660319616:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.471919:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adcab lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9377 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.471942:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adcab lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9377 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.471953:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845037.472004:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641662720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3150us (3399us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.472010:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 863 00000100:00100000:0.0:1788845037.476624:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.478534:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641662848 00000100:00100000:2.0:1788845037.478548:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641662848 00000100:00100000:2.0:1788845037.478777:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 864 00000100:00100000:2.0:1788845037.478789:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641662848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.478808:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.478828:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adcb2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a937e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.478911:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660319744:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.479002:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845037.481918:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660319744:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845037.481954:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adcb2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a937e expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.481973:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287adcb2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a937e expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.481984:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033800, rc 0) 00000100:00100000:0.0:1788845037.482027:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641662848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3241us (5415us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845037.482036:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 864 00000100:00100000:3.0:1788845037.485031:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.485144:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641662976 00000100:00100000:1.0:1788845037.485153:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641662976 00002000:00100000:1.0:1788845037.485205:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967480 x1875739641662976/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 504/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.485217:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967480 with x1875739641662976 00000100:00100000:1.0:1788845037.485253:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 849 00000100:00100000:1.0:1788845037.485263:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641662976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.498007:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.498583:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641662976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4617us (4859us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.498598:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 849 00002000:00100000:1.0:1788845037.498615:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967480 x1875739641662976/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 504/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.498632:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967480 with x1875739641662976 00000100:00100000:3.0:1788845037.498783:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641663104 00000100:00100000:3.0:1788845037.498791:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641663104 00000100:00100000:2.0:1788845037.498930:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 865 00000100:00100000:2.0:1788845037.498939:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641663104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.498955:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.498971:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287adcb9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9385 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.499031:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660319872:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.499121:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.501967:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660319872:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845037.502009:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287adcb9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9385 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.502035:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287adcb9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9385 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.502050:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845037.502121:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641663104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3182us (4122us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.502130:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 865 00000100:00100000:3.0:1788845037.507826:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.507946:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641663232 00000100:00100000:1.0:1788845037.507956:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641663232 00000100:00100000:1.0:1788845037.508010:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 866 00000100:00100000:1.0:1788845037.508020:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641663232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.508039:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.508058:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adcc0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a938c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.508124:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660320000:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.508218:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.511083:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660320000:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845037.511124:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adcc0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a938c expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.511149:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adcc0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a938c expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.511162:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:2.0:1788845037.511270:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641663232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3250us (3453us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.511279:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 866 00000100:00100000:2.0:1788845037.515229:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.515546:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641663360 00000100:00100000:3.0:1788845037.515557:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641663360 00002000:00100000:3.0:1788845037.515610:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657b80 x1875739641663360/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845037.515621:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657b80 with x1875739641663360 00000100:00100000:3.0:1788845037.515667:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 850 00000100:00100000:3.0:1788845037.515677:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641663360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845037.515970:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641663360:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (750us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845037.515978:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 850 00002000:00100000:3.0:1788845037.515985:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657b80 x1875739641663360/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845037.515993:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657b80 with x1875739641663360 00000100:00100000:2.0:1788845037.519315:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.519610:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641663488 00000100:00100000:3.0:1788845037.519620:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641663488 00000100:00100000:3.0:1788845037.519744:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 867 00000100:00100000:3.0:1788845037.519755:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641663488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.519807:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.519828:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287adcc7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9393 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.519912:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660320128:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845037.520100:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.524128:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660320128:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845037.524172:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287adcc7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9393 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.524198:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287adcc7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9393 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.524212:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845037.524323:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641663488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4569us (5017us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845037.524333:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 867 00000100:00100000:0.0:1788845037.529059:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.529143:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641663616 00000100:00100000:2.0:1788845037.529153:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641663616 00000100:00100000:2.0:1788845037.529250:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 868 00000100:00100000:2.0:1788845037.529272:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641663616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.529302:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.529320:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287adcce lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a939a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.529390:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660320256:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.529478:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845037.532614:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660320256:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845037.532722:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287adcce lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a939a expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.532749:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287adcce lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a939a expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.532775:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:0.0:1788845037.532842:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641663616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3571us (3792us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845037.532851:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 868 00000100:00100000:3.0:1788845037.536049:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.536139:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641663744 00000100:00100000:2.0:1788845037.536151:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641663744 00002000:00100000:2.0:1788845037.536205:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967b80 x1875739641663744/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.536217:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967b80 with x1875739641663744 00000100:00100000:2.0:1788845037.536264:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 851 00000100:00100000:2.0:1788845037.536276:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641663744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.536760:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641663744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 484us (718us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.536777:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 851 00002000:00100000:2.0:1788845037.536785:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967b80 x1875739641663744/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.536792:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967b80 with x1875739641663744 00000100:00100000:2.0:1788845037.541031:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.541125:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641663872 00000100:00100000:1.0:1788845037.541135:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641663872 00000100:00100000:1.0:1788845037.541193:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 869 00000100:00100000:1.0:1788845037.541203:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641663872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.541221:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.541239:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcd5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93a1 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.541314:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660320384:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.541555:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.544071:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660320384:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.544114:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcd5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93a1 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.544149:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcd5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93a1 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.544161:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845037.544223:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641663872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3021us (3202us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.544231:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 869 00000100:00100000:3.0:1788845037.549119:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.549232:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641664000 00000100:00100000:1.0:1788845037.549243:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641664000 00000100:00100000:1.0:1788845037.549302:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 870 00000100:00100000:1.0:1788845037.549312:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641664000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.549332:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.549351:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adcdc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93a8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.549424:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660320512:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.549504:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.553618:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660320512:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.553672:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adcdc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93a8 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.553699:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287adcdc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93a8 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.553717:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845037.553785:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641664000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4474us (4675us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.553795:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 870 00000100:00100000:3.0:1788845037.557140:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.557604:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641664128 00000100:00100000:0.0:1788845037.557614:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641664128 00002000:00100000:0.0:1788845037.557662:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966d80 x1875739641664128/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.557670:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966d80 with x1875739641664128 00000100:00100000:0.0:1788845037.557724:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 852 00000100:00100000:0.0:1788845037.557732:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641664128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845037.557982:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641664128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (850us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.557987:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 852 00002000:00100000:0.0:1788845037.557993:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966d80 x1875739641664128/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.557998:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966d80 with x1875739641664128 00000100:00100000:3.0:1788845037.560329:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.560717:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641664256 00000100:00100000:1.0:1788845037.560728:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641664256 00000100:00100000:1.0:1788845037.560792:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 871 00000100:00100000:1.0:1788845037.560802:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641664256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.560820:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.560835:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adce3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93af expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.560918:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660320640:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.561001:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.564991:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660320640:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.565034:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adce3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93af expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.565058:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adce3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93af expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.565072:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845037.565196:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641664256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4395us (4878us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.565205:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 871 00000100:00100000:3.0:1788845037.570238:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.571177:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641664384 00000100:00100000:2.0:1788845037.571188:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641664384 00002000:00100000:2.0:1788845037.571240:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4e00 x1875739641664384/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.571250:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4e00 with x1875739641664384 00000100:00100000:2.0:1788845037.571548:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 853 00000100:00100000:2.0:1788845037.571559:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641664384:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845037.577275:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 578129 -> 786432 00000004:00010000:1.0:1788845037.577298:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00000004:00010000:1.0:1788845037.577301:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00000004:00010000:1.0:1788845037.577304:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00000004:00010000:1.0:1788845037.577306:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:1.0:1788845037.578951:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2370:x1875739641664384:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 7385us (8713us total) trans 4294967748 rc 0/0 00000100:00100000:1.0:1788845037.578964:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 853 00002000:00100000:1.0:1788845037.578980:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4e00 x1875739641664384/t4294967748(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/448 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.578990:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4e00 with x1875739641664384 00000100:00100000:0.0:1788845037.584213:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.584446:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641664512 00000100:00100000:2.0:1788845037.584457:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641664512 00000100:00100000:2.0:1788845037.584563:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1253 00000100:00100000:2.0:1788845037.584575:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641664512:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845037.629657:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967748 is committed 00000001:00080000:1.0:1788845037.629686:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967741 is committed 00000100:00100000:3.0:1788845037.630015:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641664512:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 45436us (45805us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845037.630029:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1253 00000100:00100000:2.0:1788845037.634161:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.634252:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641664640 00000100:00100000:0.0:1788845037.634262:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641664640 00000100:00100000:0.0:1788845037.634335:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 872 00000100:00100000:0.0:1788845037.634346:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641664640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.634558:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.634580:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287adcea lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93b6 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.634648:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660320768:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845037.634786:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845037.637970:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660320768:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845037.638012:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287adcea lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93b6 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.638037:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287adcea lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93b6 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.638052:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:0.0:1788845037.638149:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641664640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3804us (3996us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845037.638158:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 872 00000100:00100000:2.0:1788845037.642161:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.642251:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641664768 00000100:00100000:1.0:1788845037.642269:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641664768 00000100:00100000:1.0:1788845037.642338:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 873 00000100:00100000:1.0:1788845037.642349:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641664768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.642378:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.642395:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcf1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93bd expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.642453:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660320896:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.642533:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.645732:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660320896:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.645783:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcf1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93bd expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.645811:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcf1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93bd expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.645824:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845037.645938:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641664768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3591us (3788us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.645947:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 873 00000100:00100000:2.0:1788845037.649683:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.649835:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641665024 00000100:00100000:0.0:1788845037.649846:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641665024 00000100:00100000:0.0:1788845037.649939:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1254 00000100:00100000:0.0:1788845037.649950:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641665024:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845037.650005:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845037.650033:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adbc4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92b3 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.650054:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.650059:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845037.650335:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.650350:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.650475:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.650512:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.650532:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 9 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.650548:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660321024:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845037.650632:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.655821:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.655962:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641665152 00000100:00100000:1.0:1788845037.655972:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641665152 00010000:00010000:1.0:1788845037.656013:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 10 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.656056:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 430 00000100:00100000:1.0:1788845037.656066:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641665152:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845037.656081:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845037.656088:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 10 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.656106:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 10 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.656128:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.656133:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.656143:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.656151:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.656171:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845037.656233:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641665152:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 162us (417us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.656241:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 430 00010000:00010000:0.0:1788845037.657209:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287adbd2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92ba expref: 9 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.657233:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660321024:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845037.657268:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.657277:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.657292:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845037.657592:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967749 00000004:00010000:0.0:1788845037.657657:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:0.0:1788845037.657661:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00000004:00010000:0.0:1788845037.657664:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:0.0:1788845037.657666:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00000004:00010000:0.0:1788845037.657669:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 712 00000004:00010000:0.0:1788845037.657677:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007f130ed0 reply state 000000000d8d7807 transno 4294967750 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.657687:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845037.657848:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641665024:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 7898us (8173us total) trans 4294967750 rc 0/0 00000100:00100000:0.0:1788845037.657856:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1254 00000020:00080000:0.0:1788845037.657861:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788845037.669186:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845037.670813:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.670833:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.670842:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.670848:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02030c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845037.670853:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.670861:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.670880:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00000100:00100000:1.0:1788845037.670971:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641665280 00000100:00100000:1.0:1788845037.670981:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641665280 00010000:00010000:0.0:1788845037.671215:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845037.671221:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.671228:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287adcf8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.671818:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 874 00000100:00100000:1.0:1788845037.671833:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641665280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.671855:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.671874:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93c4 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.671934:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.671943:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 8) 00010000:00010000:1.0:1788845037.671947:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 8) 00010000:00010000:1.0:1788845037.671953:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.671992:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.672013:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 10 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.672029:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660321152:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845037.672121:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.676190:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.676262:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641665408 00000100:00100000:0.0:1788845037.676271:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641665408 00002000:00100000:0.0:1788845037.676337:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1699840->1703239) req@ffff94a03f657800 x1875739641665408/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 3888/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.676349:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f657800 with x1875739641665408 ext(1699840->1703239) 00010000:00010000:0.0:1788845037.676366:0:13321:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 11 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.676388:0:13321:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 11 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845037.676403:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f657800 00000100:00100000:0.0:1788845037.676439:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 22 00000100:00100000:0.0:1788845037.676449:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641665408:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845037.676961:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641665408:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 511us (780us total) trans 4294967447 rc 0/0 00000100:00100000:0.0:1788845037.676970:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 22 00002000:00100000:0.0:1788845037.676977:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1699840->1703239) req@ffff94a03f657800 x1875739641665408/t4294967447(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 3888/448 e 0 to 0 dl 1788845048 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.676985:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f657800 with x1875739641665408 ext(1699840->1703239) 00010000:00010000:0.0:1788845037.677002:0:13321:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 13 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.677018:0:13321:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 13 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845037.677025:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f657800 00000100:00100000:1.0:1788845037.677930:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660321152:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845037.677961:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.677980:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845037.678024:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641665280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6195us (8849us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.678031:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 874 00000100:00100000:2.0:1788845037.683452:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.683680:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641665536 00000100:00100000:1.0:1788845037.683691:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641665536 00010000:00010000:1.0:1788845037.683733:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 13 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.683782:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 431 00000100:00100000:1.0:1788845037.683792:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641665536:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845037.683807:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845037.683830:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 13 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.683857:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 13 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.683921:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287adbf5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1699840->1703935) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a92dd expref: 13 pid: 6682 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.683941:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 8) 00010000:00010000:1.0:1788845037.683946:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 8) 00010000:00010000:1.0:1788845037.683955:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.683966:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.684004:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.684019:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660321280:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845037.684098:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.688247:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.688369:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660321280:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845037.688394:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845037.688451:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641665536:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4660us (5010us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.688462:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 431 00000100:00100000:3.0:1788845037.689032:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641665664 00000100:00100000:3.0:1788845037.689040:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641665664 00002000:00010000:3.0:1788845037.689076:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1674591->18446744073709551615) 00002000:00100000:3.0:1788845037.689081:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036081180 with x1875739641665664 ext(1674591->18446744073709551615) 00002000:00010000:3.0:1788845037.689092:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036081180. 00000100:00100000:3.0:1788845037.689117:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 437 00000100:00100000:3.0:1788845037.689126:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641665664:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845037.689551:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1703240 -> 1674591 00002000:00010000:3.0:1788845037.689555:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00002000:00010000:3.0:1788845037.689558:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00002000:00010000:3.0:1788845037.689560:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00002000:00010000:3.0:1788845037.689563:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:3.0:1788845037.689621:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641665664:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 495us (1384us total) trans 4294967448 rc 0/0 00000100:00100000:3.0:1788845037.689629:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 437 00002000:00010000:3.0:1788845037.689634:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1674591->18446744073709551615) 00002000:00100000:3.0:1788845037.689636:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036081180 with x1875739641665664 ext(1674591->18446744073709551615) 00002000:00010000:3.0:1788845037.689641:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036081180. 00000100:00100000:2.0:1788845037.694723:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.695209:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641665792 00000100:00100000:3.0:1788845037.695219:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641665792 00000100:00100000:2.0:1788845037.695787:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1255 00000100:00100000:2.0:1788845037.695798:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641665792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.695824:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.695846:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287add06 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93cb expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.695956:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.695971:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.695982:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845037.696006:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1674591 00000004:00200000:2.0:1788845037.696011:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845037.696029:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.696054:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287add06 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.696070:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.696079:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.696081:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845037.696085:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:2.0:1788845037.696152:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641665792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 356us (1442us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845037.696160:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1255 00000100:00100000:3.0:1788845037.699095:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.699214:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641665920 00000100:00100000:0.0:1788845037.699224:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641665920 00000100:00100000:0.0:1788845037.699493:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1256 00000100:00100000:0.0:1788845037.699505:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641665920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.699528:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.699549:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add14 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93d2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.699588:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.699628:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660321408:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845037.699728:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.702175:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.702234:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641666048 00000100:00100000:3.0:1788845037.702244:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641666048 00000100:00100000:3.0:1788845037.702314:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 875 00000100:00100000:3.0:1788845037.702323:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641666048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.702341:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.702369:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287add1b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93d9 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.702430:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660321536:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845037.702584:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845037.704757:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845036 -> 1788845037 00000100:00100000:2.0:1788845037.704799:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660321536:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845037.704834:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287add1b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93d9 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.704860:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003ffc065c/0xb2187301287add1b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93d9 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.704873:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191ea00, rc 0) 00000100:00100000:2.0:1788845037.704954:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641666048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2631us (2789us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.704963:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 875 00000004:00010000:0.0:1788845037.706076:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845037 00000100:00100000:0.0:1788845037.706102:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660321408:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845037.706120:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:0.0:1788845037.706136:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add14 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93d2 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.706152:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.706155:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845037.706159:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add14 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93d2 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.706172:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:0.0:1788845037.706224:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641665920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6722us (7138us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845037.711071:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.711125:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.711218:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641666176 00000100:00100000:2.0:1788845037.711228:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641666176 00000100:00100000:2.0:1788845037.711294:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1257 00000100:00100000:2.0:1788845037.711304:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641666176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.711325:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.711345:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287add22 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93e0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.711394:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.711436:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036083480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660321664:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.711518:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.712028:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641666304 00000100:00100000:1.0:1788845037.712035:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641666304 00000100:00100000:1.0:1788845037.712074:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 876 00000100:00100000:1.0:1788845037.712081:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641666304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.712090:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.712102:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287add29 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93e7 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.712142:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660321792:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.712187:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.715269:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036083480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660321664:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845037.715308:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:2.0:1788845037.715330:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287add22 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93e0 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.715352:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.715355:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845037.715487:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287add22 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93e0 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.715504:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:2.0:1788845037.715560:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641666176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4257us (4433us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.715569:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1257 00000100:00100000:1.0:1788845037.715700:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660321792:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.715722:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287add29 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93e7 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.715739:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287add29 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93e7 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.715749:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845037.715786:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641666304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3705us (4724us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.715793:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 876 00000100:00100000:0.0:1788845037.717458:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1256 00000100:00100000:1.0:1788845037.724071:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.724145:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641666432 00000100:00100000:0.0:1788845037.724156:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641666432 00000100:00100000:0.0:1788845037.724213:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1258 00000100:00100000:0.0:1788845037.724222:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641666432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.724248:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.724288:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93ee expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.724322:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.724333:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.724439:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.724458:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 15 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.724473:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660321920:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845037.724617:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845037.732720:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660321920:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845037.732777:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.732799:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.732802:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845037.732807:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.732822:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:0.0:1788845037.732935:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641666432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8713us (8874us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.732945:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1258 00000100:00100000:2.0:1788845037.735209:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.735420:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641666560 00000100:00100000:1.0:1788845037.735430:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641666560 00010000:00010000:1.0:1788845037.735471:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 16 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.735520:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 432 00000100:00100000:1.0:1788845037.735530:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d4b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641666560:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845037.735546:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845037.735577:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 16 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.735596:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 16 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.735617:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adc8f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a935b expref: 16 pid: 5841 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.735633:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.735651:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.735662:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.735671:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.735708:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.735727:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660322048:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845037.735823:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.741575:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.741662:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641666688 00000100:00100000:2.0:1788845037.741672:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641666688 00002000:00100000:2.0:1788845037.741723:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9f80 x1875739641666688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.741735:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9f80 with x1875739641666688 00000100:00100000:2.0:1788845037.741780:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 854 00000100:00100000:2.0:1788845037.741789:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641666688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.742044:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641666688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 255us (480us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845037.742052:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 854 00002000:00100000:2.0:1788845037.742059:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9f80 x1875739641666688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/4536 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.742067:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9f80 with x1875739641666688 00000100:00100000:0.0:1788845037.743001:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660322048:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845037.743024:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845037.743071:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d4b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641666560:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7542us (7873us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.743081:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 432 00000100:00100000:1.0:1788845037.747091:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.747202:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641666816 00000100:00100000:0.0:1788845037.747212:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641666816 00002000:00100000:0.0:1788845037.747262:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9180 x1875739641666816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.747273:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9180 with x1875739641666816 00000100:00100000:0.0:1788845037.747318:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 855 00000100:00100000:0.0:1788845037.747327:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641666816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845037.747546:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641666816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 219us (465us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845037.747554:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 855 00002000:00100000:0.0:1788845037.747561:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9180 x1875739641666816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/4536 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845037.747569:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9180 with x1875739641666816 00000100:00100000:2.0:1788845037.751952:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.752257:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641666944 00000100:00100000:3.0:1788845037.752268:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641666944 00000100:00100000:3.0:1788845037.752340:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 877 00000100:00100000:3.0:1788845037.752352:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641666944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.752372:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.752391:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287add37 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93f5 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.752451:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660322176:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845037.752540:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.755986:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660322176:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845037.756027:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287add37 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93f5 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.756054:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287add37 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93f5 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.756068:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845037.756139:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641666944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3789us (4198us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845037.756149:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 877 00000100:00100000:2.0:1788845037.762404:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.762500:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641667072 00000100:00100000:1.0:1788845037.762510:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641667072 00000100:00100000:1.0:1788845037.762571:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 878 00000100:00100000:1.0:1788845037.762581:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641667072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.762602:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.762621:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287add3e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a93fc expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.762694:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660322304:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.762785:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097869c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.765200:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660322304:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.765242:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287add3e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93fc expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.765279:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287add3e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93fc expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.765301:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845037.765370:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641667072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2790us (2976us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.765379:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 878 00000100:00100000:2.0:1788845037.768707:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.768810:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641667200 00000100:00100000:3.0:1788845037.768821:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641667200 00000100:00100000:3.0:1788845037.768926:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1259 00000100:00100000:3.0:1788845037.768938:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641667200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.768964:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.768985:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9403 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.769016:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.769026:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.769061:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.769079:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 10 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.769093:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660322432:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845037.769174:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.773245:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.773406:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641667328 00000100:00100000:1.0:1788845037.773416:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641667328 00002000:00100000:1.0:1788845037.773468:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082a00 x1875739641667328/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.773478:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082a00 with x1875739641667328 00010000:00010000:1.0:1788845037.773487:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 11 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.773505:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 11 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845037.773513:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a036082a00 00000100:00100000:1.0:1788845037.773553:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 29 00000100:00100000:1.0:1788845037.773563:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641667328:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845037.773807:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660322432:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845037.773844:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.773866:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845037.773868:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845037.773872:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.773918:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:3.0:1788845037.773951:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641667200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5016us (5254us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845037.773958:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1259 00000004:00010000:3.0:1788845037.779000:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:3.0:1788845037.779010:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00000004:00010000:3.0:1788845037.779012:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00000004:00010000:3.0:1788845037.779015:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00000004:00010000:3.0:1788845037.779017:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845037.779095:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641667328:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5529us (5856us total) trans 4294967751 rc 0/0 00000100:00100000:3.0:1788845037.779105:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 29 00002000:00100000:3.0:1788845037.779118:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082a00 x1875739641667328/t4294967751(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/448 e 0 to 0 dl 1788845048 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845037.779128:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082a00 with x1875739641667328 00010000:00010000:3.0:1788845037.779139:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.779160:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845037.779169:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a036082a00 00000100:00100000:2.0:1788845037.783039:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.783232:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641667456 00000100:00100000:0.0:1788845037.783244:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641667456 00010000:00010000:0.0:1788845037.783306:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.783359:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 433 00000100:00100000:0.0:1788845037.783369:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641667456:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845037.783384:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845037.783412:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.783430:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.783452:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add30 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93ee expref: 13 pid: 5839 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.783469:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.783475:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.783487:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.783496:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.783533:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.783550:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660322560:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845037.783634:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.787874:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.788001:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641667584 00000100:00100000:1.0:1788845037.788011:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641667584 00000100:00100000:1.0:1788845037.788080:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 856 00000100:00100000:1.0:1788845037.788090:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641667584:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845037.788336:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:1.0:1788845037.788340:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00000004:00010000:1.0:1788845037.788343:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845037.788345:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00000004:00010000:1.0:1788845037.788347:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845037.788414:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641667584:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 325us (549us total) trans 4294967752 rc 0/0 00000100:00100000:1.0:1788845037.788423:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 856 00000100:00100000:0.0:1788845037.789109:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660322560:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845037.789131:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845037.789167:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641667456:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5799us (6139us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.789177:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 433 00000100:00100000:2.0:1788845037.793153:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.793252:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641667712 00000100:00100000:3.0:1788845037.793272:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641667712 00000100:00100000:3.0:1788845037.793344:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1260 00000100:00100000:3.0:1788845037.793354:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641667712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.793380:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.793401:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add4c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a940a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.793496:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287add53 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.793512:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287add53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.793525:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287add53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845037.793552:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1674591 00000004:00200000:3.0:1788845037.793557:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845037.793574:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287add53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.793602:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add4c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a940a expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.793619:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287add53 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a940a expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.793633:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845037.793647:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845037.793651:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191fe00, rc 0) 00000100:00100000:3.0:1788845037.794487:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641667712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1130us (1348us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845037.794496:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1260 00000100:00100000:1.0:1788845037.800109:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.800213:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641667840 00000100:00100000:0.0:1788845037.800225:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641667840 00000100:00100000:0.0:1788845037.800373:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1261 00000100:00100000:0.0:1788845037.800386:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641667840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.800414:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.800438:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9411 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.800467:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.800476:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.800510:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.800528:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 17 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.800544:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660322688:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845037.800630:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.804639:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.804734:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660322688:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845037.804780:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.804801:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.804803:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845037.804807:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.804817:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:0.0:1788845037.804906:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641667840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4522us (4808us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.804914:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1261 00000100:00100000:3.0:1788845037.805486:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641667968 00000100:00100000:3.0:1788845037.805496:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641667968 00010000:00010000:3.0:1788845037.805534:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 18 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.805582:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 434 00000100:00100000:3.0:1788845037.805595:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641667968:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845037.805611:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845037.805649:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 18 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.805665:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 18 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.805687:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f200/0xb2187301287add45 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9403 expref: 18 pid: 11036 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.805702:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845037.805707:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.805717:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.805726:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.805768:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.805782:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660322816:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845037.806191:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.810499:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660322816:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845037.810528:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845037.810587:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641667968:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4994us (5960us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845037.810600:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 434 00000100:00100000:2.0:1788845037.812685:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.812805:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641668096 00000100:00100000:0.0:1788845037.812815:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641668096 00000100:00100000:0.0:1788845037.812921:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 857 00000100:00100000:0.0:1788845037.812931:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641668096:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845037.813166:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:0.0:1788845037.813170:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00000004:00010000:0.0:1788845037.813172:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:0.0:1788845037.813174:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00000004:00010000:0.0:1788845037.813176:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845037.813245:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641668096:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 315us (568us total) trans 4294967753 rc 0/0 00000100:00100000:0.0:1788845037.813252:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 857 00000100:00100000:1.0:1788845037.818175:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.818324:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641668224 00000100:00100000:2.0:1788845037.818334:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641668224 00000100:00100000:2.0:1788845037.818392:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 879 00000100:00100000:2.0:1788845037.818403:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641668224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.818425:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.818448:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287add61 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9418 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.818509:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02362d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660322944:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.818634:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.822156:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02362d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660322944:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.822199:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287add61 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9418 expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.822224:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287add61 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9418 expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.822237:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:1.0:1788845037.822305:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641668224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3903us (4138us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.822313:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 879 00000100:00100000:1.0:1788845037.825668:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.825797:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641668352 00000100:00100000:3.0:1788845037.825807:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641668352 00000100:00100000:3.0:1788845037.825872:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 880 00000100:00100000:3.0:1788845037.825908:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641668352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.825927:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.825944:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287add68 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a941f expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.826001:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660323072:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845037.826086:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.829506:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660323072:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.829552:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287add68 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a941f expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.829577:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f087238/0xb2187301287add68 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a941f expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.829590:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f200, rc 0) 00000100:00100000:1.0:1788845037.829660:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641668352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3778us (4000us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.829669:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 880 00000100:00100000:2.0:1788845037.833623:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.833756:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641668480 00000100:00100000:1.0:1788845037.833774:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641668480 00000100:00100000:1.0:1788845037.833833:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1262 00000100:00100000:1.0:1788845037.833843:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641668480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.833863:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.833926:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287add6f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9426 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.833965:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.834008:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660323200:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.834085:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097869c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788845037.836294:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845037 00000100:00100000:1.0:1788845037.836326:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660323200:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845037.836349:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:1.0:1788845037.836367:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287add6f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9426 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.836387:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.836390:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845037.836394:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287add6f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9426 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.836415:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845037.836475:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641668480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2633us (2861us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.836484:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1262 00000100:00100000:0.0:1788845037.840572:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.840666:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641668608 00000100:00100000:3.0:1788845037.840677:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641668608 00000100:00100000:3.0:1788845037.840736:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1263 00000100:00100000:3.0:1788845037.840747:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641668608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.840777:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.840798:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a942d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.840828:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.840838:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.840870:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.840928:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 13 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.840942:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660323328:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845037.841021:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.843827:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660323328:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845037.843950:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.843975:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845037.843979:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845037.843982:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.843992:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f600, rc 0) 00000100:00100000:3.0:1788845037.844040:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641668608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3295us (3479us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845037.844048:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1263 00000100:00100000:1.0:1788845037.846944:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.847038:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641668736 00000100:00100000:0.0:1788845037.847049:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641668736 00010000:00010000:0.0:1788845037.847089:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 14 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.847133:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 435 00000100:00100000:0.0:1788845037.847143:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641668736:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845037.847160:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845037.847188:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 14 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.847205:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 14 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.847229:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287add5a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9411 expref: 14 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.847246:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.847253:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.847280:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.847302:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.847340:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.847354:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660323456:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845037.847435:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.851153:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660323456:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845037.851181:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845037.851239:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641668736:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4097us (4304us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.851252:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 435 00000100:00100000:3.0:1788845037.854566:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.854661:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641668864 00000100:00100000:1.0:1788845037.854677:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641668864 00002000:00100000:1.0:1788845037.854735:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5f80 x1875739641668864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.854746:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5f80 with x1875739641668864 00000100:00100000:1.0:1788845037.854856:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 858 00000100:00100000:1.0:1788845037.854866:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641668864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845037.864377:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2367:x1875739641668864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 9500us (9815us total) trans 0 rc 65536/65536 00000100:00100000:1.0:1788845037.864391:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 858 00002000:00100000:1.0:1788845037.864404:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5f80 x1875739641668864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 488/440 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845037.864414:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5f80 with x1875739641668864 00000100:00100000:2.0:1788845037.870747:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.872412:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641668992 00000100:00100000:3.0:1788845037.872433:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641668992 00002000:00100000:3.0:1788845037.872506:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02363b80 x1875739641668992/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 16872/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845037.872521:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02363b80 with x1875739641668992 00000100:00100000:3.0:1788845037.872617:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 859 00000100:00100000:3.0:1788845037.872630:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641668992:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845037.873393:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:3.0:1788845037.873403:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00000004:00010000:3.0:1788845037.873406:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00000004:00010000:3.0:1788845037.873408:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00000004:00010000:3.0:1788845037.873411:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845037.873501:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641668992:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 868us (2760us total) trans 4294967754 rc 0/0 00000100:00100000:3.0:1788845037.873517:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 859 00002000:00100000:3.0:1788845037.873534:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02363b80 x1875739641668992/t4294967754(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:143/0 lens 16872/448 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845037.873544:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02363b80 with x1875739641668992 00000100:00100000:2.0:1788845037.876656:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.877425:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641669120 00000100:00100000:3.0:1788845037.877437:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641669120 00000100:00100000:0.0:1788845037.877623:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1264 00000100:00100000:0.0:1788845037.877646:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641669120:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845037.907048:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967753 is committed 00000001:00080000:1.0:1788845037.907064:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967750 is committed 00000001:00080000:1.0:1788845037.907071:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967752 is committed 00000001:00080000:1.0:1788845037.907076:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967754 is committed 00000100:00100000:0.0:1788845037.907230:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641669120:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 29591us (30587us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.907242:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1264 00000100:00100000:0.0:1788845037.912242:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.912725:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.913075:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641669376 00000100:00100000:3.0:1788845037.913086:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641669376 00000100:00100000:2.0:1788845037.913198:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641669248 00000100:00100000:2.0:1788845037.913202:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641669248 00000100:00100000:2.0:1788845037.913250:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1265 00000100:00100000:2.0:1788845037.913260:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641669248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.913293:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.913317:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287add7d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9434 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.913354:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.913395:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660323584:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.913471:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.915756:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 881 00000100:00100000:1.0:1788845037.915790:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641669376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.915811:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.915922:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287add84 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a943b expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.915982:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660323712:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.916115:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097869c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.919066:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660323712:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.919106:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287add84 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a943b expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.919132:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287add84 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a943b expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.919145:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845037.919210:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641669376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3431us (6485us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.919219:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 881 00000004:00010000:2.0:1788845037.920192:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845037 00000100:00100000:2.0:1788845037.920218:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660323584:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845037.920239:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:2.0:1788845037.920254:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287add7d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9434 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.920272:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.920275:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845037.920291:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287add7d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9434 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.920305:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:2.0:1788845037.920353:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641669248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7095us (8121us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.920361:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1265 00000100:00100000:0.0:1788845037.925501:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.926171:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.926240:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641669632 00000100:00100000:2.0:1788845037.926250:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641669632 00000100:00100000:2.0:1788845037.926322:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 882 00000100:00100000:2.0:1788845037.926332:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641669632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.926352:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.926369:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287add8b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9449 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.926431:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660323840:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.926516:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.927740:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641669504 00000100:00100000:1.0:1788845037.927752:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641669504 00000100:00100000:1.0:1788845037.927878:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1266 00000100:00100000:1.0:1788845037.927922:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641669504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.927940:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.927958:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287add92 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9442 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.927990:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.928025:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660323968:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845037.928108:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097869c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.930327:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660323840:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845037.930365:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287add8b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9449 expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.930388:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287add8b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9449 expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.930403:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032800, rc 0) 00000100:00100000:2.0:1788845037.930459:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641669632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4128us (4289us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.930467:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 882 00000100:00100000:1.0:1788845037.934106:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660323968:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845037.934137:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:1.0:1788845037.934158:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287add92 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9442 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.934180:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.934182:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845037.934187:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287add92 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9442 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.934202:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845037.934275:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641669504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6355us (8782us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.934292:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1266 00000100:00100000:1.0:1788845037.938099:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.938236:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641669760 00000100:00100000:0.0:1788845037.938246:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641669760 00000100:00100000:0.0:1788845037.938324:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1267 00000100:00100000:0.0:1788845037.938334:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641669760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.938356:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.938376:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9450 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.938412:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.938423:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.938460:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.938480:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 14 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.938493:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660324096:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845037.938582:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.942611:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.942746:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641669888 00000100:00100000:1.0:1788845037.942756:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641669888 00010000:00010000:1.0:1788845037.942806:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 15 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.942859:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 436 00000100:00100000:1.0:1788845037.942868:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641669888:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845037.942915:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845037.942944:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 15 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.942961:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 15 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.942983:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.942990:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.943001:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.943010:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.943047:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.943061:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660324224:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845037.943139:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097869c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788845037.944133:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191f600/0xb2187301287add76 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a942d expref: 15 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.944153:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660324096:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845037.944179:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845037.944192:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845037.944195:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845037.944198:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:0.0:1788845037.944251:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641669760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5918us (6161us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845037.944269:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1267 00000100:00100000:3.0:1788845037.948655:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.948799:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641670016 00000100:00100000:2.0:1788845037.948809:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641670016 00002000:00100000:2.0:1788845037.948871:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b7800 x1875739641670016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.948921:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7800 with x1875739641670016 00000100:00100000:2.0:1788845037.948952:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 860 00000100:00100000:2.0:1788845037.948961:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641670016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.949239:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641670016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 278us (600us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845037.949247:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 860 00002000:00100000:2.0:1788845037.949254:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b7800 x1875739641670016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/4536 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.949261:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7800 with x1875739641670016 00000100:00100000:1.0:1788845037.949478:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660324224:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845037.949499:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845037.949538:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641669888:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6670us (6940us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.949546:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 436 00000100:00100000:3.0:1788845037.953482:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.953586:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641670144 00000100:00100000:2.0:1788845037.953597:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641670144 00002000:00100000:2.0:1788845037.953657:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4a80 x1875739641670144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.953668:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4a80 with x1875739641670144 00000100:00100000:2.0:1788845037.953705:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 861 00000100:00100000:2.0:1788845037.953717:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641670144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845037.954025:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641670144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 309us (550us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845037.954033:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 861 00002000:00100000:2.0:1788845037.954041:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b4a80 x1875739641670144/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/4536 e 0 to 0 dl 1788845048 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.954048:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b4a80 with x1875739641670144 00000100:00100000:3.0:1788845037.958066:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845037.958166:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641670272 00000100:00100000:0.0:1788845037.958177:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641670272 00000100:00100000:0.0:1788845037.958238:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 883 00000100:00100000:0.0:1788845037.958248:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641670272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845037.958265:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845037.958295:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bdc72b9b/0xb2187301287adda0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9457 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845037.958356:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660324352:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845037.958438:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.961876:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660324352:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845037.961951:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bdc72b9b/0xb2187301287adda0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9457 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.961975:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bdc72b9b/0xb2187301287adda0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9457 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.961987:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031400, rc 0) 00000100:00100000:1.0:1788845037.962047:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641670272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3801us (3991us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845037.962057:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 883 00000100:00100000:0.0:1788845037.965633:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845037.965723:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641670400 00000100:00100000:3.0:1788845037.965734:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641670400 00000100:00100000:3.0:1788845037.965802:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 884 00000100:00100000:3.0:1788845037.965813:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641670400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845037.965841:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845037.965860:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287adda7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a945e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845037.965966:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660324480:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845037.966041:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f02e4cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845037.969567:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660324480:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845037.969611:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287adda7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a945e expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.969653:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a771f60/0xb2187301287adda7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a945e expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845037.969668:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f0191f800, rc 0) 00000100:00100000:3.0:1788845037.969729:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641670400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3918us (4105us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845037.969738:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 884 00000100:00100000:0.0:1788845037.975356:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.975500:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641670528 00000100:00100000:2.0:1788845037.975511:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641670528 00000100:00100000:2.0:1788845037.975573:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1268 00000100:00100000:2.0:1788845037.975584:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641670528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.975604:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.975627:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287addae lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9465 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.975792:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.975944:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660324608:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.976020:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.978726:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660324608:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845037.978754:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:2.0:1788845037.978955:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287addae lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9465 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.978975:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.978979:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845037.978984:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287addae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9465 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.979000:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:2.0:1788845037.979051:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641670528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3469us (3704us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.979061:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1268 00000100:00100000:0.0:1788845037.982295:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.982504:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641670656 00000100:00100000:2.0:1788845037.982515:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641670656 00000100:00100000:2.0:1788845037.982571:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1269 00000100:00100000:2.0:1788845037.982582:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641670656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845037.982605:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845037.982626:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287addb5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a946c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.982677:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845037.982722:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660324736:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845037.982818:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845037.985823:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660324736:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845037.985861:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:2.0:1788845037.985880:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287addb5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a946c expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.985930:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845037.985934:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845037.985939:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287addb5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a946c expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.985954:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845037.986004:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641670656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3423us (3724us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845037.986013:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1269 00000100:00100000:3.0:1788845037.989110:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845037.989213:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641670784 00000100:00100000:1.0:1788845037.989223:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641670784 00000100:00100000:1.0:1788845037.989276:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1270 00000100:00100000:1.0:1788845037.989297:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641670784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845037.989319:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845037.989340:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9473 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.989375:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.989386:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.989418:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.989436:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 10 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845037.989450:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035328e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660324864:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845037.989532:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097869c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845037.992074:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035328e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660324864:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845037.992111:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.992132:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845037.992136:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845037.992141:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845037.992152:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845037.992210:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641670784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2915us (3110us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845037.992220:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1270 00000100:00100000:0.0:1788845037.994965:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845037.995066:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641670912 00000100:00100000:2.0:1788845037.995077:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641670912 00002000:00100000:2.0:1788845037.995125:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9a680 x1875739641670912/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/0 e 0 to 0 dl 1788845048 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845037.995135:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9a680 with x1875739641670912 00010000:00010000:2.0:1788845037.995143:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 11 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845037.995163:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 11 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845037.995171:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f01f9a680 00000100:00100000:2.0:1788845037.995204:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 30 00000100:00100000:2.0:1788845037.995215:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641670912:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845038.000819:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:2.0:1788845038.000984:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00000004:00010000:2.0:1788845038.000987:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00000004:00010000:2.0:1788845038.000990:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00000004:00010000:2.0:1788845038.000993:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:2.0:1788845038.001074:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641670912:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5856us (6113us total) trans 4294967755 rc 0/0 00000100:00100000:2.0:1788845038.001085:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 30 00002000:00100000:2.0:1788845038.001099:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9a680 x1875739641670912/t4294967755(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:143/0 lens 488/448 e 0 to 0 dl 1788845048 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845038.001109:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9a680 with x1875739641670912 00010000:00010000:2.0:1788845038.001119:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 13 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.001140:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 13 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845038.001149:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f01f9a680 00000100:00100000:3.0:1788845038.004390:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.005006:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641671040 00000100:00100000:2.0:1788845038.005016:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641671040 00010000:00010000:2.0:1788845038.005057:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 13 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.005101:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 437 00000100:00100000:2.0:1788845038.005111:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641671040:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845038.005125:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845038.005152:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 13 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.005169:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 13 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.005196:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287add99 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9450 expref: 13 pid: 11036 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.005215:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.005221:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.005234:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.005243:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.005400:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.005414:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660324992:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845038.005499:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.009435:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.010158:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660324992:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845038.010186:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845038.010238:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641671040:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5127us (5856us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.010249:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 437 00000100:00100000:3.0:1788845038.012132:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641671168 00000100:00100000:3.0:1788845038.012146:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641671168 00000100:00100000:0.0:1788845038.012627:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 862 00000100:00100000:0.0:1788845038.012714:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641671168:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845038.013245:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:0.0:1788845038.013250:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00000004:00010000:0.0:1788845038.013489:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:0.0:1788845038.013493:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845037 -> 1788845037 00000004:00010000:0.0:1788845038.013496:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845038.013640:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641671168:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 920us (4205us total) trans 4294967756 rc 0/0 00000100:00100000:0.0:1788845038.013653:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 862 00000100:00100000:1.0:1788845038.017985:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.018018:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.018109:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641671424 00000100:00100000:2.0:1788845038.018119:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641671424 00000100:00100000:2.0:1788845038.018177:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 885 00000100:00100000:2.0:1788845038.018186:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641671424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.018204:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.018222:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287addc3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9481 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.018289:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660325120:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.018372:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.019242:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641671296 00000100:00100000:0.0:1788845038.019251:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641671296 00000100:00100000:0.0:1788845038.019309:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1271 00000100:00100000:0.0:1788845038.019318:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641671296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.019336:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.019353:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287addca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a947a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.019384:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.019415:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660325248:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.019475:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00002000:00010000:2.0:1788845038.023260:0:6684:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845037 -> 1788845038 00000100:00100000:2.0:1788845038.023298:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660325120:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845038.023333:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287addc3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9481 expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.023358:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287addc3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9481 expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.023373:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845038.023427:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641671424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5241us (5407us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.023435:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 885 00000004:00010000:0.0:1788845038.024569:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845037 00000004:00010000:0.0:1788845038.024577:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845037 -> 1788845038 00000100:00100000:0.0:1788845038.024597:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660325248:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.024613:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:0.0:1788845038.024628:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287addca lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a947a expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.024654:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.024657:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.024662:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287addca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a947a expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.024681:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845038.024733:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641671296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5416us (6759us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.024741:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1271 00000100:00100000:1.0:1788845038.030540:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.030587:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.030668:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641671552 00000100:00100000:0.0:1788845038.030678:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641671552 00000100:00100000:0.0:1788845038.030748:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1272 00000100:00100000:0.0:1788845038.030768:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641671552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.030789:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.030810:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287addd1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9488 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.030850:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.030931:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660325376:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.031005:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.031602:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641671680 00000100:00100000:2.0:1788845038.031610:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641671680 00000100:00100000:2.0:1788845038.031657:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 886 00000100:00100000:2.0:1788845038.031666:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641671680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.031678:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.031691:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287addd8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a948f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.031731:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660325504:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.031786:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.035783:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660325376:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.035810:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:0.0:1788845038.035839:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287addd1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9488 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.035859:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.035862:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.035868:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287addd1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9488 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.035916:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:0.0:1788845038.035980:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641671552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5222us (5391us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.035989:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1272 00000100:00100000:2.0:1788845038.036560:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660325504:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845038.036585:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287addd8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a948f expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.036602:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287addd8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a948f expref: 13 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.036613:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845038.036661:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641671680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4996us (6130us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.036668:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 886 00000100:00100000:1.0:1788845038.041994:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.042130:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641671808 00000100:00100000:2.0:1788845038.042141:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641671808 00000100:00100000:2.0:1788845038.042203:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1273 00000100:00100000:2.0:1788845038.042215:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641671808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.042236:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.042268:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9496 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.042307:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.042321:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.042355:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.042373:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 15 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.042387:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660325632:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845038.042467:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.047841:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660325632:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845038.047879:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.047931:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.047934:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845038.047938:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.047949:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:2.0:1788845038.048005:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641671808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5793us (6021us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.048014:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1273 00000100:00100000:3.0:1788845038.052960:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.053188:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641671936 00000100:00100000:1.0:1788845038.053199:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641671936 00010000:00010000:1.0:1788845038.053251:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 16 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.053314:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 438 00000100:00100000:1.0:1788845038.053324:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641671936:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845038.053340:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845038.053366:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 16 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.053384:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 16 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.053407:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287addbc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9473 expref: 16 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.053423:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.053430:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.053441:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.053456:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.053496:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.053509:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660325760:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845038.053605:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097869c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.059606:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660325760:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845038.059643:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845038.059709:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641671936:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6385us (6758us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845038.059721:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 438 00000100:00100000:2.0:1788845038.064865:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.064992:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641672064 00000100:00100000:1.0:1788845038.065002:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641672064 00000100:00100000:1.0:1788845038.065059:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 887 00000100:00100000:1.0:1788845038.065068:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641672064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.065086:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.065104:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287adde6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a949d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.065168:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660325888:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.065254:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097866c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.069115:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660325888:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845038.069172:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287adde6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a949d expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.069198:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287adde6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a949d expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.069219:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845038.069289:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641672064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4221us (4432us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.069299:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 887 00000100:00100000:3.0:1788845038.079086:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.079264:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641672192 00000100:00100000:0.0:1788845038.079274:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641672192 00002000:00100000:0.0:1788845038.079335:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6680 x1875739641672192/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 504/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.079346:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6680 with x1875739641672192 00000100:00100000:0.0:1788845038.079390:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 863 00000100:00100000:0.0:1788845038.079400:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641672192:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845038.100649:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641672192:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 21235us (21560us total) trans 0 rc 786432/786432 00000100:00100000:1.0:1788845038.100663:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 863 00002000:00100000:1.0:1788845038.100677:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6680 x1875739641672192/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 504/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 786432/786432 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.100687:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6680 with x1875739641672192 00000100:00100000:3.0:1788845038.105261:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.105651:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641672320 00000100:00100000:0.0:1788845038.105662:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641672320 00002000:00100000:0.0:1788845038.105714:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5f80 x1875739641672320/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.105724:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5f80 with x1875739641672320 00000100:00100000:0.0:1788845038.105774:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 864 00000100:00100000:0.0:1788845038.105785:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641672320:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845038.106103:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641672320:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 319us (851us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.106110:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 864 00002000:00100000:0.0:1788845038.106118:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5f80 x1875739641672320/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.106131:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5f80 with x1875739641672320 00000100:00100000:0.0:1788845038.110065:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.110329:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641672448 00000100:00100000:2.0:1788845038.110340:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641672448 00002000:00100000:2.0:1788845038.110393:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654380 x1875739641672448/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845038.110405:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654380 with x1875739641672448 00000100:00100000:2.0:1788845038.110445:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 865 00000100:00100000:2.0:1788845038.110455:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641672448:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845038.110716:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641672448:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (672us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.110724:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 865 00002000:00100000:2.0:1788845038.110730:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654380 x1875739641672448/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845038.110738:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654380 with x1875739641672448 00000100:00100000:1.0:1788845038.114976:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.115334:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641672576 00000100:00100000:0.0:1788845038.115344:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641672576 00002000:00100000:0.0:1788845038.115395:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48a80 x1875739641672576/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.115407:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48a80 with x1875739641672576 00000100:00100000:0.0:1788845038.115448:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 866 00000100:00100000:0.0:1788845038.115458:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641672576:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845038.115750:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641672576:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 292us (783us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.115767:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 866 00002000:00100000:0.0:1788845038.115774:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01c48a80 x1875739641672576/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.115782:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c48a80 with x1875739641672576 00000100:00100000:3.0:1788845038.121453:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.121562:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641672704 00000100:00100000:1.0:1788845038.121572:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641672704 00000100:00100000:1.0:1788845038.121643:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 888 00000100:00100000:1.0:1788845038.121654:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641672704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.121671:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.121689:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287added lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94a4 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.121746:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660326016:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.122231:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097866c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.125022:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660326016:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845038.125209:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287added lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94a4 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.125240:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287added lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94a4 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.125255:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845038.125317:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641672704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3664us (3872us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.125325:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 888 00000100:00100000:3.0:1788845038.129456:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.129555:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641672832 00000100:00100000:2.0:1788845038.129566:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641672832 00000100:00100000:2.0:1788845038.129623:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 889 00000100:00100000:2.0:1788845038.129643:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641672832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.129662:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.129680:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000028aee44e/0xb2187301287addf4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94ab expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.129737:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660326144:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.130611:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.135111:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660326144:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845038.135288:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000028aee44e/0xb2187301287addf4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ab expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.135318:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028aee44e/0xb2187301287addf4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ab expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.135334:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033000, rc 0) 00000100:00100000:2.0:1788845038.135403:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641672832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5771us (5957us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.135412:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 889 00000100:00100000:3.0:1788845038.139467:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.139587:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641672960 00000100:00100000:2.0:1788845038.139597:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641672960 00000100:00100000:2.0:1788845038.139668:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 890 00000100:00100000:2.0:1788845038.139678:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641672960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.139696:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.139713:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94b2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.139746:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.139764:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(9 8) 00010000:00010000:2.0:1788845038.139769:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 8) 00010000:00010000:2.0:1788845038.139774:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1589248->1626111] (req 1589248->1626111) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.139809:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 10 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.139827:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 10 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.139841:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660326272:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845038.139948:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.143288:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660326272:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845038.143325:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1589248->1626111] (req 1589248->1626111) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.143346:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845038.143405:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641672960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3728us (3948us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.143415:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 890 00000100:00100000:3.0:1788845038.148334:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.148438:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641673088 00000100:00100000:1.0:1788845038.148449:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641673088 00010000:00010000:1.0:1788845038.148491:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 11 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.148538:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 439 00000100:00100000:1.0:1788845038.148548:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a031d48c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641673088:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845038.148565:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845038.148589:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 11 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.148609:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 11 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.148646:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287adcff lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93c4 expref: 11 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.148670:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 8) 00010000:00010000:1.0:1788845038.148674:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 8) 00010000:00010000:1.0:1788845038.148683:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 12 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.148693:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 12 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.148732:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 12 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.148746:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660326400:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845038.148831:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097866c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.153706:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.153796:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641673216 00000100:00100000:0.0:1788845038.153807:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641673216 00000100:00100000:0.0:1788845038.153915:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 438 00000100:00100000:0.0:1788845038.153925:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641673216:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:0.0:1788845038.154706:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1674591 -> 1674591 00002000:00010000:0.0:1788845038.154713:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845037 -> 1788845037 00002000:00010000:0.0:1788845038.154716:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00002000:00010000:0.0:1788845038.154718:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845038 -> 1788845037 00002000:00010000:0.0:1788845038.154721:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:0.0:1788845038.154811:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641673216:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 885us (1114us total) trans 4294967449 rc 0/0 00000100:00100000:0.0:1788845038.154819:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 438 00000100:00100000:1.0:1788845038.156100:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660326400:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845038.156124:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845038.156351:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a031d48c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641673088:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7803us (8026us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.156362:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 439 00010000:00010000:2.0:1788845038.162505:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845038.168793:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.169021:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641673344 00000100:00100000:1.0:1788845038.169032:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641673344 00000100:00100000:1.0:1788845038.169105:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 891 00000100:00100000:1.0:1788845038.169116:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641673344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.169267:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.169300:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94b9 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.169334:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.169344:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 8) 00010000:00010000:1.0:1788845038.169348:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 8) 00010000:00010000:1.0:1788845038.169354:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [2052096->2080767] (req 2052096->2080767) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.169392:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.169411:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 13 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.169426:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660326528:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845038.169515:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.173564:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.174013:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641673472 00000100:00100000:0.0:1788845038.174023:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641673472 00010000:00010000:0.0:1788845038.174074:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 14 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.174118:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 440 00000100:00100000:0.0:1788845038.174128:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641673472:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845038.174152:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845038.174178:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 14 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.174197:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 14 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.174222:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 8) 00010000:00010000:0.0:1788845038.174227:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 8) 00010000:00010000:0.0:1788845038.174235:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.174245:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.174305:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.174321:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660326656:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845038.174419:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00010000:00010000:1.0:1788845038.175488:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287addfb lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1589248->1626111) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b2 expref: 14 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.175507:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660326528:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845038.175536:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.175552:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845038.175611:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:14891:x1875739641673344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6496us (6830us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.175617:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 891 00000100:00100000:0.0:1788845038.180861:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660326656:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845038.180925:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845038.180999:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641673472:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6870us (7442us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.181012:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 440 00000100:00100000:2.0:1788845038.187576:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.187945:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641673600 00000100:00100000:1.0:1788845038.187956:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641673600 00000100:00100000:1.0:1788845038.188015:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 892 00000100:00100000:1.0:1788845038.188026:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641673600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.188056:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.188077:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade09 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94c0 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.188375:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660326784:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.188468:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786c00 going to sleep for 11 seconds 00002000:00010000:1.0:1788845038.191590:0:6685:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1674591 -> 2077771 00002000:00010000:1.0:1788845038.191600:0:6685:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845037 -> 1788845038 00002000:00010000:1.0:1788845038.191602:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845036 -> 1788845037 00002000:00010000:1.0:1788845038.191605:0:6685:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845037 -> 1788845038 00000100:00100000:1.0:1788845038.191639:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660326784:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845038.191677:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade09 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94c0 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.191702:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade09 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94c0 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.191721:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845038.191790:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641673600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3766us (4222us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.191799:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 892 00000100:00100000:3.0:1788845038.196348:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.196502:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641673728 00000100:00100000:2.0:1788845038.196513:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641673728 00000100:00100000:2.0:1788845038.196570:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 893 00000100:00100000:2.0:1788845038.196580:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641673728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.196598:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.196616:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ade10 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94c7 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.196707:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02362300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660326912:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.196814:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.202323:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02362300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660326912:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845038.202371:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ade10 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94c7 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.202399:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ade10 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94c7 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.202414:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033800, rc 0) 00000100:00100000:1.0:1788845038.202478:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641673728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5898us (6138us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.202485:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 893 00000100:00100000:3.0:1788845038.208997:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.209311:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641673856 00000100:00100000:2.0:1788845038.209321:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641673856 00000100:00100000:2.0:1788845038.209377:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 894 00000100:00100000:2.0:1788845038.209388:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641673856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.209408:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.209426:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade17 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94ce expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.209488:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660327040:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.209575:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.213430:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660327040:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845038.213474:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade17 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ce expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.213499:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade17 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ce expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.213513:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033a00, rc 0) 00000100:00100000:2.0:1788845038.213574:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641673856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4187us (4587us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.213582:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 894 00000100:00100000:3.0:1788845038.219119:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.219779:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641673984 00000100:00100000:1.0:1788845038.219790:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641673984 00000100:00100000:1.0:1788845038.220104:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 895 00000100:00100000:1.0:1788845038.220118:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034508050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641673984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.220211:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.220231:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade1e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94d5 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.220509:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660327168:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.220600:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.224714:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660327168:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845038.224764:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade1e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94d5 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.224791:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade1e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94d5 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.224805:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845038.225019:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034508050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641673984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4902us (5909us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.225028:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 895 00000100:00100000:3.0:1788845038.228458:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.228978:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641674112 00000100:00100000:1.0:1788845038.228989:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641674112 00000100:00100000:1.0:1788845038.229056:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1274 00000100:00100000:1.0:1788845038.229067:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641674112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.229093:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.229117:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ade25 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94dc expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.229166:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ade25 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94dc expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845038.229183:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:1.0:1788845038.229192:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ade25 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94dc expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.229206:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.229209:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845038.229212:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845038.229290:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641674112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 224us (840us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.229297:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1274 00000100:00100000:3.0:1788845038.237045:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.237159:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641674240 00000100:00100000:1.0:1788845038.237170:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641674240 00000100:00100000:1.0:1788845038.237230:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 896 00000100:00100000:1.0:1788845038.237240:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641674240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.237258:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.237288:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94e3 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.237319:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.237328:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 8) 00010000:00010000:1.0:1788845038.237333:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:1.0:1788845038.237338:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1101824->1126399] (req 1101824->1126399) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 14 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.237379:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.237397:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 10 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.237412:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660327296:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845038.237495:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.241107:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660327296:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845038.241160:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1101824->1126399] (req 1101824->1126399) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 14 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.241183:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845038.241247:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641674240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4007us (4211us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.241255:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 896 00000100:00100000:3.0:1788845038.242967:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.243243:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641674368 00000100:00100000:1.0:1788845038.243253:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641674368 00002000:00100000:1.0:1788845038.243313:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (2052096->2077770) req@ffff949f01fc6300 x1875739641674368/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.243323:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc6300 with x1875739641674368 ext(2052096->2077770) 00010000:00010000:1.0:1788845038.243332:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 11 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.243358:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 11 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845038.243366:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f01fc6300 00000100:00100000:1.0:1788845038.243388:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 23 00000100:00100000:1.0:1788845038.243398:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641674368:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845038.248546:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641674368:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5137us (5573us total) trans 4294967450 rc 0/0 00000100:00100000:1.0:1788845038.248561:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 23 00002000:00100000:1.0:1788845038.248574:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (2052096->2077770) req@ffff949f01fc6300 x1875739641674368/t4294967450(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/448 e 0 to 0 dl 1788845049 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.248585:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc6300 with x1875739641674368 ext(2052096->2077770) 00010000:00010000:1.0:1788845038.248595:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 13 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.248618:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 13 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845038.248627:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f01fc6300 00000100:00100000:3.0:1788845038.253521:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.253740:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641674496 00000100:00100000:2.0:1788845038.253751:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641674496 00010000:00010000:2.0:1788845038.253798:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 13 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.253850:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 441 00000100:00100000:2.0:1788845038.253860:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641674496:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845038.253876:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845038.253942:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 13 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.253960:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 13 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.253983:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ade02 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2052096->2080767) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94b9 expref: 13 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.254001:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788845038.254005:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788845038.254014:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.254025:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.254074:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.254090:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660327424:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845038.254359:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.258928:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660327424:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845038.258957:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845038.259018:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:2369:x1875739641674496:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5158us (5505us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.259029:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 441 00000100:00100000:3.0:1788845038.270630:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.270766:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641674624 00000100:00100000:1.0:1788845038.270777:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641674624 00002000:00100000:1.0:1788845038.270830:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2080767) req@ffff949f01fc6a00 x1875739641674624/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.270848:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc6a00 with x1875739641674624 ext(1048576->2080767) 00002000:00010000:1.0:1788845038.270854:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc6a00 00000100:00100000:1.0:1788845038.270929:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 439 00000100:00100000:1.0:1788845038.270939:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641674624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845038.286511:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.286619:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641674752 00000100:00100000:2.0:1788845038.286635:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641674752 00002000:00100000:2.0:1788845038.286686:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc4a80 x1875739641674752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 504/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845038.286699:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4a80 with x1875739641674752 00000100:00100000:2.0:1788845038.286739:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 867 00000100:00100000:2.0:1788845038.286750:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641674752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845038.292911:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641674624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 21937us (22256us total) trans 0 rc 1032192/1032192 00000100:00100000:1.0:1788845038.292926:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 439 00002000:00100000:1.0:1788845038.292940:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2080767) req@ffff949f01fc6a00 x1875739641674624/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 1032192/1032192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.292949:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc6a00 with x1875739641674624 ext(1048576->2080767) 00002000:00010000:1.0:1788845038.292956:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc6a00 00000100:00100000:2.0:1788845038.294363:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641674752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 7608us (7855us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.294375:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 867 00002000:00100000:2.0:1788845038.294387:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc4a80 x1875739641674752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 504/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845038.294397:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4a80 with x1875739641674752 00000100:00100000:3.0:1788845038.300521:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.300857:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641674880 00000100:00100000:2.0:1788845038.300867:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641674880 00000100:00100000:2.0:1788845038.301217:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 897 00000100:00100000:2.0:1788845038.301228:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:2369:x1875739641674880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.301247:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.301267:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94ea expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.301357:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788845038.301362:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788845038.301372:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.301389:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 13 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.301403:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033a00, rc 0) 00000100:00100000:2.0:1788845038.301560:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641674880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 333us (1049us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.301567:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 897 00000100:00100000:0.0:1788845038.306562:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.306734:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641675136 00000100:00100000:3.0:1788845038.306744:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641675136 00000100:00100000:3.0:1788845038.306861:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1275 00000100:00100000:3.0:1788845038.306872:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641675136:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845038.306962:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845038.306988:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f0191fe00/0xb2187301287add53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a940a expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.307008:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845038.307013:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845038.307251:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.307285:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.307293:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.307326:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.307344:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 12 pid: 5840 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.307358:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660327552:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845038.307432:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0016719c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.311841:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.311939:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641675264 00000100:00100000:1.0:1788845038.311950:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641675264 00010000:00010000:1.0:1788845038.311991:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 13 pid: 5840 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.312032:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 442 00000100:00100000:1.0:1788845038.312053:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641675264:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845038.312068:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845038.312075:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 13 pid: 5840 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.312093:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 13 pid: 5840 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.312117:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.312122:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.312141:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.312150:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.312169:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845038.312224:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641675264:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 182us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.312232:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 442 00010000:00010000:0.0:1788845038.314701:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287add0d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a93cb expref: 12 pid: 5840 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.314728:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660327552:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845038.314765:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.314774:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.314778:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845038.319839:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967757 00000004:00010000:0.0:1788845038.319939:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:0.0:1788845038.319944:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845037 -> 1788845038 00000004:00010000:0.0:1788845038.319946:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00000004:00010000:0.0:1788845038.319949:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00000004:00010000:0.0:1788845038.319951:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:0.0:1788845038.319963:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000626fe357 reply state 0000000057eb660a transno 4294967758 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.319974:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845038.320117:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641675136:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13244us (13562us total) trans 4294967758 rc 0/0 00000100:00100000:0.0:1788845038.320125:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1275 00000020:00080000:0.0:1788845038.320141:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845038.323278:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845038.324182:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.324195:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.324202:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.324208:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02031400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845038.324212:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.324219:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.324236:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.324239:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845038.324243:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.324249:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ade3a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.325125:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641675392 00000100:00100000:2.0:1788845038.325148:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641675392 00000100:00100000:2.0:1788845038.325213:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1276 00000100:00100000:2.0:1788845038.325225:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641675392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.325251:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.325264:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845038.325314:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ade25 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94dc expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.325328:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.325331:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845038.325345:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94f1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.325376:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.325386:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.325419:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.325437:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 11 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.325451:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660327680:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845038.325546:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.329482:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660327680:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845038.329527:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.329550:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.329553:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845038.329558:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.329569:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845038.329654:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641675392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4431us (6393us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.329664:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1276 00000100:00100000:3.0:1788845038.336485:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.336578:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641675520 00000100:00100000:0.0:1788845038.336588:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641675520 00010000:00010000:0.0:1788845038.336642:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 12 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.336683:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 443 00000100:00100000:0.0:1788845038.336694:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641675520:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845038.336713:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845038.336740:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 12 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.336768:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 12 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.336791:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287adddf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9496 expref: 12 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.336807:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.336812:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.336822:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.336830:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.336872:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.336919:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660327808:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845038.337439:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.345391:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.345513:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641675648 00000100:00100000:2.0:1788845038.345523:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641675648 00000100:00100000:2.0:1788845038.345587:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 898 00000100:00100000:2.0:1788845038.345598:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641675648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.345618:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.345642:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94f8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.345677:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.345686:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.345694:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788845038.345699:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788845038.345705:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 14 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.345740:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.345769:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 14 pid: 6684 timeout: 527 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.345791:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02360000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660327936:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845038.345909:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.345920:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 13 pid: 6685 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.345929:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660328064:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845038.345957:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.346211:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660327808:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845038.346230:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845038.346257:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641675520:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9566us (9783us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.346266:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 443 00000100:00100000:2.0:1788845038.349655:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02360000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660327936:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845038.349672:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660328064:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845038.349706:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 14 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.349726:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:2.0:1788845038.349793:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641675648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4197us (4412us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.349801:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 898 00000100:00100000:3.0:1788845038.356057:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.356101:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.356251:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641675776 00000100:00100000:1.0:1788845038.356272:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641675776 00010000:00010000:1.0:1788845038.356324:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 14 pid: 6684 timeout: 527 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.356368:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641675904 00000100:00100000:1.0:1788845038.356371:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641675904 00010000:00010000:1.0:1788845038.356385:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 14 pid: 6685 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.356408:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 444 00000100:00100000:1.0:1788845038.356417:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641675776:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845038.356434:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845038.356461:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 14 pid: 6684 timeout: 527 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.356479:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 14 pid: 6684 timeout: 527 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.356504:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ade33 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ea expref: 14 pid: 6684 timeout: 527 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.356523:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845038.356635:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 445 00000100:00100000:2.0:1788845038.356645:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641675904:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845038.356653:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845038.356665:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 14 pid: 6685 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.356677:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 14 pid: 6685 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.356691:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ade2c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1126399) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94e3 expref: 14 pid: 6685 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.356700:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788845038.356704:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788845038.356712:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.356721:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.356767:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.356782:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660328192:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845038.357904:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641675776:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 1399us (1780us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.357916:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 444 00000100:00100000:2.0:1788845038.358064:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.365615:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.365805:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641676032 00000100:00100000:1.0:1788845038.365816:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641676032 00002000:00010000:1.0:1788845038.365860:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1199403->18446744073709551615) 00002000:00100000:1.0:1788845038.365865:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360000 with x1875739641676032 ext(1199403->18446744073709551615) 00002000:00010000:1.0:1788845038.365878:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360000. 00000100:00100000:1.0:1788845038.365934:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 440 00000100:00100000:1.0:1788845038.365944:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641676032:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845038.366539:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2077771 -> 1199403 00002000:00010000:1.0:1788845038.366544:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00002000:00010000:1.0:1788845038.366546:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00002000:00010000:1.0:1788845038.366548:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00002000:00010000:1.0:1788845038.366550:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:1.0:1788845038.366606:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641676032:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 663us (1001us total) trans 4294967451 rc 0/0 00000100:00100000:1.0:1788845038.367012:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 440 00002000:00010000:1.0:1788845038.367019:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1199403->18446744073709551615) 00002000:00100000:1.0:1788845038.367022:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02360000 with x1875739641676032 ext(1199403->18446744073709551615) 00002000:00010000:1.0:1788845038.367028:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02360000. 00000100:00100000:3.0:1788845038.367364:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660328192:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845038.367387:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845038.367418:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641675904:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10774us (11316us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845038.367426:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 445 00000100:00100000:2.0:1788845038.371505:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.371595:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641676160 00000100:00100000:0.0:1788845038.371605:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641676160 00000100:00100000:0.0:1788845038.372566:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1277 00000100:00100000:0.0:1788845038.372582:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641676160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.372604:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.372638:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ade4f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a94ff expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.372729:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ade56 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.372744:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ade56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.372766:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ade56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845038.372798:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1199403 00000004:00200000:0.0:1788845038.372802:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845038.372821:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ade56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.372849:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ade4f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ff expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.372866:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ade56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ff expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.372874:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.372877:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845038.372880:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006800, rc 0) 00000100:00100000:0.0:1788845038.372972:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641676160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 396us (1477us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.372979:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1277 00000100:00100000:3.0:1788845038.379847:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.380675:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641676288 00000100:00100000:2.0:1788845038.380686:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641676288 00000100:00100000:2.0:1788845038.380801:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1278 00000100:00100000:2.0:1788845038.380812:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641676288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.380835:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.380857:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ade5d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9506 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.381021:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.381038:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.381049:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845038.381076:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1199403 00000004:00200000:2.0:1788845038.381080:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845038.381098:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.381123:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ade5d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.381205:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.381275:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.381279:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845038.381282:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:2.0:1788845038.381399:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641676288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 588us (1561us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.381407:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1278 00000100:00100000:2.0:1788845038.383693:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.383797:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641676416 00000100:00100000:3.0:1788845038.383807:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641676416 00000100:00100000:3.0:1788845038.383874:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1279 00000100:00100000:3.0:1788845038.384010:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641676416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.384031:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.384051:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ade6b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a950d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.384087:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.384250:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beaaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660328320:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845038.384869:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0016719c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.387214:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.391474:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641676544 00000004:00010000:0.0:1788845038.391474:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845037 00000100:00100000:1.0:1788845038.391489:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641676544 00000100:00100000:0.0:1788845038.391512:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beaaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660328320:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.391541:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00000100:00100000:1.0:1788845038.391555:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 899 00010000:00010000:0.0:1788845038.391559:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ade6b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a950d expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.391564:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641676544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.391580:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.391580:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.391583:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.391588:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ade6b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a950d expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.391595:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ade72 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9514 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.391602:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:0.0:1788845038.391656:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641676416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7773us (7971us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.391663:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1279 00000100:00100000:1.0:1788845038.392921:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660328448:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.393010:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786c00 going to sleep for 11 seconds 00002000:00010000:1.0:1788845038.396092:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845036 -> 1788845037 00000100:00100000:1.0:1788845038.396123:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660328448:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845038.396511:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ade72 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9514 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.396531:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ade72 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9514 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.396541:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:1.0:1788845038.396589:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641676544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5025us (9386us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.396597:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 899 00000100:00100000:0.0:1788845038.400106:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.400151:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.400278:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641676800 00000100:00100000:1.0:1788845038.400287:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641676800 00000100:00100000:1.0:1788845038.400346:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 900 00000100:00100000:1.0:1788845038.400356:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641676800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.400375:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.400391:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ade79 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9522 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.400451:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660328576:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.400523:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.400769:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641676672 00000100:00100000:2.0:1788845038.400774:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641676672 00000100:00100000:2.0:1788845038.400820:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1280 00000100:00100000:2.0:1788845038.400829:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641676672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.400839:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.400853:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ade80 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a951b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.400919:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.400946:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660328704:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.400975:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.402992:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660328576:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845038.403034:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ade79 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9522 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.403058:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ade79 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9522 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.403071:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:1.0:1788845038.403122:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641676800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2766us (3025us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.403140:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 900 00000100:00100000:3.0:1788845038.404605:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660328704:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845038.404628:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:3.0:1788845038.404656:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ade80 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a951b expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.404674:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845038.404676:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845038.404679:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ade80 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a951b expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.404689:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033800, rc 0) 00000100:00100000:3.0:1788845038.404733:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641676672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3905us (4579us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.404740:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1280 00000100:00100000:2.0:1788845038.410562:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.410661:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641676928 00000100:00100000:3.0:1788845038.410671:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641676928 00002000:00100000:3.0:1788845038.410721:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02362300 x1875739641676928/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845038.410733:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02362300 with x1875739641676928 00000100:00100000:3.0:1788845038.410777:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 868 00000100:00100000:3.0:1788845038.410787:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641676928:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845038.411268:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641676928:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 481us (716us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845038.411277:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 868 00002000:00100000:3.0:1788845038.411284:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02362300 x1875739641676928/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/4536 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845038.411300:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02362300 with x1875739641676928 00000100:00100000:2.0:1788845038.413701:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.413781:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641677056 00000100:00100000:1.0:1788845038.413791:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641677056 00002000:00100000:1.0:1788845038.413851:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02362d80 x1875739641677056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.413862:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02362d80 with x1875739641677056 00000100:00100000:1.0:1788845038.413926:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 869 00000100:00100000:1.0:1788845038.413935:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641677056:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845038.414153:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641677056:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 219us (461us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845038.414161:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 869 00002000:00100000:1.0:1788845038.414168:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02362d80 x1875739641677056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/4536 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.414176:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02362d80 with x1875739641677056 00000100:00100000:0.0:1788845038.417502:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.417607:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641677184 00000100:00100000:2.0:1788845038.417616:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641677184 00000100:00100000:2.0:1788845038.417675:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1281 00000100:00100000:2.0:1788845038.417685:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641677184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.417706:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.417726:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ade87 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9529 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.417781:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.417825:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660328832:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.417941:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.419052:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.419122:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641677312 00000100:00100000:3.0:1788845038.419130:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641677312 00000100:00100000:3.0:1788845038.419176:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 901 00000100:00100000:3.0:1788845038.419185:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641677312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.419199:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.419213:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ade8e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9530 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.419267:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beaad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660328960:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845038.419332:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0016719c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.420714:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660328832:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845038.420738:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:2.0:1788845038.420764:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ade87 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9529 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.420783:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.420786:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845038.420791:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ade87 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9529 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.420806:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845038.420863:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641677184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3178us (3370us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.420870:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1281 00000100:00100000:0.0:1788845038.421002:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beaad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660328960:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.421024:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ade8e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9530 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.421039:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ade8e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9530 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.421049:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:0.0:1788845038.421073:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641677312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1890us (2026us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.421080:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 901 00000100:00100000:0.0:1788845038.423726:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.423799:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641677440 00000100:00100000:1.0:1788845038.423808:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641677440 00000100:00100000:1.0:1788845038.423861:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1282 00000100:00100000:1.0:1788845038.423872:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641677440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.423916:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.423936:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ade95 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9537 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.423973:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.424006:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660329088:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.424170:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.425136:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.425199:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641677568 00000100:00100000:2.0:1788845038.425205:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641677568 00000100:00100000:2.0:1788845038.425252:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 902 00000100:00100000:2.0:1788845038.425270:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641677568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.425282:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.425295:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ade9c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a953e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.425339:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660329216:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.425388:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.427991:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660329088:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845038.428017:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:1.0:1788845038.428036:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ade95 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9537 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.428058:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.428061:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845038.428066:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ade95 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9537 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.428082:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:1.0:1788845038.428145:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641677440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4274us (4427us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.428154:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1282 00000100:00100000:2.0:1788845038.433465:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660329216:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845038.433509:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ade9c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a953e expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.433534:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ade9c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a953e expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.433547:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:2.0:1788845038.433608:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641677568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8339us (8475us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.433617:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 902 00000100:00100000:3.0:1788845038.437223:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.437382:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641677696 00000100:00100000:1.0:1788845038.437392:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641677696 00000100:00100000:1.0:1788845038.437447:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 903 00000100:00100000:1.0:1788845038.437457:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641677696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.437477:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.437495:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9545 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.437522:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.437528:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:1.0:1788845038.437531:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 8) 00010000:00010000:1.0:1788845038.437535:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1069056->1101823] (req 1069056->1101823) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.437563:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.437578:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 13 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.437589:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660329344:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845038.437665:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.440194:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660329344:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845038.440243:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1069056->1101823] (req 1069056->1101823) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.440274:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:1.0:1788845038.440334:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641677696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2878us (3120us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.440343:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 903 00000100:00100000:0.0:1788845038.443028:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.443204:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641677824 00000100:00100000:3.0:1788845038.443215:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641677824 00010000:00010000:3.0:1788845038.443260:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 14 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.443411:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 446 00000100:00100000:3.0:1788845038.443422:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019e6f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641677824:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845038.443438:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845038.443464:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 14 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.443483:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 14 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.443505:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ade48 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f8 expref: 14 pid: 7983 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.443523:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 8) 00010000:00010000:3.0:1788845038.443528:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 8) 00010000:00010000:3.0:1788845038.443537:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.443547:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.443589:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.443604:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660329472:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845038.443771:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0016719c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.446677:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660329472:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845038.446707:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845038.446767:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019e6f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641677824:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3346us (3747us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845038.446778:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 446 00000100:00100000:2.0:1788845038.450242:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.450456:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641677952 00000100:00100000:0.0:1788845038.450467:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641677952 00002000:00100000:0.0:1788845038.450515:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1069056->1073151) req@ffff949f02363480 x1875739641677952/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.450525:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02363480 with x1875739641677952 ext(1069056->1073151) 00002000:00010000:0.0:1788845038.450531:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02363480 00000100:00100000:0.0:1788845038.450563:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 441 00000100:00100000:0.0:1788845038.450573:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641677952:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845038.451053:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641677952:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 477us (816us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845038.451061:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 441 00002000:00100000:0.0:1788845038.451069:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1069056->1073151) req@ffff949f02363480 x1875739641677952/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/4536 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.451077:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02363480 with x1875739641677952 ext(1069056->1073151) 00002000:00010000:0.0:1788845038.451081:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02363480 00000100:00100000:0.0:1788845038.457017:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.457249:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641678080 00000100:00100000:1.0:1788845038.457260:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641678080 00002000:00100000:1.0:1788845038.457325:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1097728->1101823) req@ffff94a03f654e00 x1875739641678080/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.457337:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f654e00 with x1875739641678080 ext(1097728->1101823) 00002000:00010000:1.0:1788845038.457343:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f654e00 00000100:00100000:1.0:1788845038.457379:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 442 00000100:00100000:1.0:1788845038.457389:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641678080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845038.457621:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641678080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (615us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845038.457638:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 442 00002000:00100000:1.0:1788845038.457645:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1097728->1101823) req@ffff94a03f654e00 x1875739641678080/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/4536 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.457653:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f654e00 with x1875739641678080 ext(1097728->1101823) 00002000:00010000:1.0:1788845038.457657:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f654e00 00000100:00100000:0.0:1788845038.463092:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.463237:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641678208 00000100:00100000:3.0:1788845038.463248:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641678208 00000100:00100000:3.0:1788845038.463324:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1283 00000100:00100000:3.0:1788845038.463335:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641678208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.463359:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.463382:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287adeaa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a954c expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.463420:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.463462:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660329600:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845038.463550:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.467424:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660329600:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.467467:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:0.0:1788845038.467487:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287adeaa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a954c expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.467507:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.467511:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.467516:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287adeaa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a954c expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.467537:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:0.0:1788845038.467605:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641678208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4272us (4524us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.467614:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1283 00000100:00100000:1.0:1788845038.477341:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.478398:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641678336 00000100:00100000:3.0:1788845038.478410:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641678336 00000100:00100000:2.0:1788845038.478715:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1284 00000100:00100000:2.0:1788845038.478726:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641678336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.478756:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.478777:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287adeb1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9553 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.478817:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.478873:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660329728:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.478987:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.481796:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660329728:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.481835:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:0.0:1788845038.481857:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287adeb1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9553 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.481879:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.481913:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.481919:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287adeb1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9553 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.481937:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033000, rc 0) 00000100:00100000:0.0:1788845038.481995:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641678336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3269us (4662us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.482005:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1284 00000100:00100000:3.0:1788845038.485644:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.485880:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641678464 00000100:00100000:2.0:1788845038.485929:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641678464 00000100:00100000:2.0:1788845038.485980:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 904 00000100:00100000:2.0:1788845038.485990:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641678464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.486010:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.486029:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287adeb8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a955a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.486098:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660329856:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.486186:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.488935:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660329856:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845038.488974:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287adeb8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a955a expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.488998:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287adeb8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a955a expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.489019:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:3.0:1788845038.489081:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641678464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3092us (3457us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.489090:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 904 00000100:00100000:2.0:1788845038.494163:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.494553:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641678592 00000100:00100000:0.0:1788845038.494563:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641678592 00000100:00100000:0.0:1788845038.494631:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 905 00000100:00100000:0.0:1788845038.494643:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641678592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.494662:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.494682:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287adebf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9561 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.494760:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660329984:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.494853:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.497474:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660329984:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.497520:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287adebf lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9561 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.497548:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287adebf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9561 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.497575:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845038.497647:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641678592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3004us (3495us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.497657:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 905 00000100:00100000:2.0:1788845038.502329:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.503251:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641678720 00000100:00100000:0.0:1788845038.503355:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641678720 00000100:00100000:2.0:1788845038.503514:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1285 00000100:00100000:2.0:1788845038.503525:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641678720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.503544:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.503566:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9568 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.503596:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.503605:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.503697:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.503714:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 17 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.503729:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660330112:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845038.503924:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.508949:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.509953:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660330112:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845038.509991:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.510013:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.510017:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845038.510022:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.510046:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845038.510101:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641678720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6577us (7783us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.510111:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1285 00000100:00100000:0.0:1788845038.510455:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641678848 00000100:00100000:0.0:1788845038.510463:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641678848 00002000:00100000:0.0:1788845038.510512:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f776d80 x1875739641678848/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 16872/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.510522:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f776d80 with x1875739641678848 00010000:00010000:0.0:1788845038.510529:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 18 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.510548:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 18 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845038.510555:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a03f776d80 00000100:00100000:0.0:1788845038.510593:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 31 00000100:00100000:0.0:1788845038.510603:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641678848:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845038.511063:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:0.0:1788845038.511069:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00000004:00010000:0.0:1788845038.511072:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845037 -> 1788845036 00000004:00010000:0.0:1788845038.511074:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00000004:00010000:0.0:1788845038.511076:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845038.511117:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641678848:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 515us (2176us total) trans 4294967759 rc 0/0 00000100:00100000:0.0:1788845038.511124:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 31 00002000:00100000:0.0:1788845038.511143:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f776d80 x1875739641678848/t4294967759(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 16872/448 e 0 to 0 dl 1788845049 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.511151:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f776d80 with x1875739641678848 00010000:00010000:0.0:1788845038.511157:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 20 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.511169:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 20 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845038.511175:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a03f776d80 00000100:00100000:3.0:1788845038.514959:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.516980:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641678976 00000100:00100000:2.0:1788845038.516994:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641678976 00010000:00010000:2.0:1788845038.517205:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 20 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.517316:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 447 00000100:00100000:2.0:1788845038.517327:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2367:x1875739641678976:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845038.517342:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845038.517370:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 20 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.517388:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 20 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.517411:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ade41 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a94f1 expref: 20 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.517428:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.517435:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.517446:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.517455:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.517497:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.517512:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660330240:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845038.517610:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.524832:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660330240:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845038.524861:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845038.524930:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2367:x1875739641678976:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7605us (9979us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.524941:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 447 00000100:00100000:1.0:1788845038.530088:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.530229:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641679104 00000100:00100000:2.0:1788845038.530241:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641679104 00002000:00100000:2.0:1788845038.530313:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8fb80 x1875739641679104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845038.530325:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8fb80 with x1875739641679104 00000100:00100000:2.0:1788845038.530365:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 870 00000100:00100000:2.0:1788845038.530376:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641679104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845038.538870:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.538954:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641679232 00000100:00100000:0.0:1788845038.538964:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641679232 00002000:00100000:0.0:1788845038.539016:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1200127) req@ffff949f01d8f100 x1875739641679232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 504/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.539028:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8f100 with x1875739641679232 ext(1048576->1200127) 00002000:00010000:0.0:1788845038.539045:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8f100 00000100:00100000:0.0:1788845038.539080:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 443 00000100:00100000:0.0:1788845038.539090:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641679232:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845038.555213:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641679232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 16112us (16340us total) trans 0 rc 118784/118784 00000100:00100000:0.0:1788845038.555228:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 443 00002000:00100000:0.0:1788845038.555243:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1200127) req@ffff949f01d8f100 x1875739641679232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 504/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 118784/118784 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.555256:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8f100 with x1875739641679232 ext(1048576->1200127) 00002000:00010000:0.0:1788845038.555262:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8f100 00000100:00100000:3.0:1788845038.557441:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641679104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 27056us (27354us total) trans 0 rc 786432/786432 00000100:00100000:3.0:1788845038.557458:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 870 00002000:00100000:3.0:1788845038.557472:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01d8fb80 x1875739641679104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 786432/786432 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845038.557483:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01d8fb80 with x1875739641679104 00000100:00100000:1.0:1788845038.563959:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.564257:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641679360 00000100:00100000:2.0:1788845038.564277:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641679360 00000100:00100000:2.0:1788845038.564334:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1286 00000100:00100000:2.0:1788845038.564345:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641679360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.564371:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.564393:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a956f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.564424:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.564432:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.564465:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.564484:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 12 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.564498:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660330368:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845038.564805:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.571078:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.571204:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641679488 00000100:00100000:1.0:1788845038.571205:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660330368:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845038.571211:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641679488 00010000:00010000:1.0:1788845038.571238:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.571253:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 13 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.571276:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.571280:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845038.571285:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.571294:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 448 00010000:00010000:1.0:1788845038.571295:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:0.0:1788845038.571302:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641679488:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845038.571316:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845038.571341:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 13 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.571349:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641679360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7007us (7403us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.571356:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1286 00010000:00010000:0.0:1788845038.571357:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 13 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.571379:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287adec6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9568 expref: 13 pid: 5841 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.571392:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.571396:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.571406:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.571414:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.571447:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.571461:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660330496:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845038.572432:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.580674:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.580759:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641679616 00000100:00100000:0.0:1788845038.580768:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641679616 00000100:00100000:0.0:1788845038.580844:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 871 00000100:00100000:0.0:1788845038.580855:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641679616:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845038.581206:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:0.0:1788845038.581211:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00000004:00010000:0.0:1788845038.581224:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:0.0:1788845038.581227:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00000004:00010000:0.0:1788845038.581229:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845038.581307:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641679616:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 453us (643us total) trans 4294967760 rc 0/0 00000100:00100000:0.0:1788845038.581315:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 871 00000100:00100000:1.0:1788845038.582957:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660330496:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845038.582989:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845038.583118:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641679488:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11815us (12050us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.583371:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 448 00000100:00100000:2.0:1788845038.590333:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.591490:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641679744 00000100:00100000:0.0:1788845038.591502:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641679744 00000100:00100000:0.0:1788845038.591572:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1287 00000100:00100000:0.0:1788845038.591584:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641679744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.591607:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.591689:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aded4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9576 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.591729:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.591783:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660330624:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.592155:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000004:00010000:1.0:1788845038.597598:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845037 00000100:00100000:1.0:1788845038.597627:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660330624:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845038.597646:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:1.0:1788845038.597662:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aded4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9576 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.597678:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.597680:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845038.597684:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aded4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9576 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.597697:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006600, rc 0) 00000100:00100000:1.0:1788845038.597804:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641679744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6222us (7482us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.597811:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1287 00000100:00100000:1.0:1788845038.602088:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.602166:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641679872 00000100:00100000:0.0:1788845038.602176:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641679872 00000100:00100000:0.0:1788845038.602242:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1288 00000100:00100000:0.0:1788845038.602252:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641679872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.602284:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.602313:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287adedb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a957d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.602346:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.602384:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660330752:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.602468:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.605708:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660330752:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.605737:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:0.0:1788845038.605770:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287adedb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a957d expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.605792:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.605795:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.605799:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287adedb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a957d expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.605815:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845038.610691:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641679872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8432us (8604us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.610703:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1288 00000100:00100000:3.0:1788845038.613516:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.613657:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641680000 00000100:00100000:0.0:1788845038.613668:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641680000 00000100:00100000:0.0:1788845038.613732:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 906 00000100:00100000:0.0:1788845038.613742:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.613769:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.613787:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287adee2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9584 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.613859:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660330880:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.613961:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00002000:00010000:0.0:1788845038.617085:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845037 -> 1788845038 00000100:00100000:0.0:1788845038.617118:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660330880:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.617165:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287adee2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9584 expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.617190:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287adee2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9584 expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.617204:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845038.617276:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3528us (3761us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.617285:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 906 00000100:00100000:1.0:1788845038.621928:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.622017:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641680128 00000100:00100000:0.0:1788845038.622027:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641680128 00000100:00100000:0.0:1788845038.622086:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 907 00000100:00100000:0.0:1788845038.622098:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.622119:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.622151:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adee9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a958b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.622210:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660331008:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.622760:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.627002:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660331008:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.627046:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adee9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a958b expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.627072:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adee9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a958b expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.627086:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845038.627157:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5062us (5277us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.627167:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 907 00000100:00100000:1.0:1788845038.631086:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.631173:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641680256 00000100:00100000:3.0:1788845038.631184:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641680256 00000100:00100000:3.0:1788845038.631248:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 908 00000100:00100000:3.0:1788845038.631259:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.631279:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.631309:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adef0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9592 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.631380:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660331136:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845038.631459:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.634433:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660331136:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845038.634469:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adef0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9592 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.634489:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adef0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9592 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.634500:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:3.0:1788845038.634552:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3295us (3478us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.634561:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 908 00000100:00100000:1.0:1788845038.638725:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.640772:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641680384 00000100:00100000:3.0:1788845038.640789:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641680384 00000100:00100000:2.0:1788845038.641009:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 909 00000100:00100000:2.0:1788845038.641021:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.641042:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.641061:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287adef7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9599 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.641132:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660331264:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.641243:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.644577:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660331264:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845038.644620:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287adef7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9599 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.644658:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287adef7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9599 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.644672:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:2.0:1788845038.644735:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3715us (6019us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.644757:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 909 00000100:00100000:1.0:1788845038.651970:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.652061:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641680512 00000100:00100000:0.0:1788845038.652072:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641680512 00000100:00100000:0.0:1788845038.652132:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 910 00000100:00100000:0.0:1788845038.652142:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.652162:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.652180:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adefe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95a0 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.652240:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660331392:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.652333:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.657247:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660331392:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.657299:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adefe lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95a0 expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.657324:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adefe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95a0 expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.657341:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845038.657406:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5265us (5445us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.657415:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 910 00000100:00100000:1.0:1788845038.663122:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.663221:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641680640 00000100:00100000:0.0:1788845038.663232:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641680640 00000100:00100000:0.0:1788845038.663361:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 911 00000100:00100000:0.0:1788845038.663380:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.663402:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.663420:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adf05 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95a7 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.663484:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660331520:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.663573:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.670548:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660331520:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.670594:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adf05 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95a7 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.670618:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287adf05 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95a7 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.670645:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845038.670708:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641680640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7328us (7597us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.670717:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 911 00000100:00100000:1.0:1788845038.677190:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.677293:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641680768 00000100:00100000:2.0:1788845038.677304:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641680768 00000100:00100000:2.0:1788845038.677364:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1289 00000100:00100000:2.0:1788845038.677374:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641680768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.677398:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.677419:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287adf0c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95ae expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.677457:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.677500:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660331648:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.677587:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845038.680560:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845037 -> 1788845038 00000100:00100000:2.0:1788845038.680593:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660331648:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845038.680619:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:2.0:1788845038.680650:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287adf0c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ae expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.680669:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.680672:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845038.680676:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287adf0c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ae expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.680692:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:2.0:1788845038.680762:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641680768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3389us (3582us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.680770:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1289 00000100:00100000:3.0:1788845038.683404:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.683555:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641680896 00000100:00100000:1.0:1788845038.683564:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641680896 00000100:00100000:1.0:1788845038.683623:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1290 00000100:00100000:1.0:1788845038.683642:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641680896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.683661:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.683681:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287adf13 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95b5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.683719:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.683773:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660331776:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.683853:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0097869c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.687189:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660331776:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845038.687217:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 786432 00010000:00010000:2.0:1788845038.687237:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287adf13 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95b5 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.687270:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.687273:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845038.687278:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287adf13 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95b5 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.687292:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:2.0:1788845038.687349:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641680896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3709us (3955us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.687357:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1290 00000100:00100000:2.0:1788845038.690436:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.690536:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641681152 00000100:00100000:1.0:1788845038.690547:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641681152 00000100:00100000:1.0:1788845038.690612:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1291 00000100:00100000:1.0:1788845038.690623:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641681152:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845038.690787:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845038.690814:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ade56 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a94ff expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.690832:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.690837:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845038.691098:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.691116:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.691123:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.691157:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.691176:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 11 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.691191:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660331904:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845038.691281:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.693815:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660331904:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845038.693858:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.696135:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.696616:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641681280 00000100:00100000:3.0:1788845038.696636:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641681280 00010000:00010000:3.0:1788845038.696683:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 12 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.696854:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 449 00000100:00100000:3.0:1788845038.696865:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641681280:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845038.696914:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845038.696923:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 12 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.696942:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 12 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.696973:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ade64 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9506 expref: 12 pid: 5839 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.696992:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845038.696997:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.697006:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.697014:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.697041:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845038.697105:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641681280:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 242us (979us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845038.697112:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 449 00010000:00010000:1.0:1788845038.697173:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.697182:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845038.697318:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967761 00000004:00010000:1.0:1788845038.697371:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 786432 00000004:00010000:1.0:1788845038.697376:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00000004:00010000:1.0:1788845038.697378:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845038 -> 1788845036 00000004:00010000:1.0:1788845038.697381:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00000004:00010000:1.0:1788845038.697383:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:1.0:1788845038.697390:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000060c0e207 reply state 00000000ad0f8bd2 transno 4294967762 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.697399:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845038.697472:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641681152:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6851us (7046us total) trans 4294967762 rc 0/0 00000100:00100000:1.0:1788845038.697480:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1291 00000020:00080000:1.0:1788845038.697483:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845038.699691:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.699717:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.699725:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.699732:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845038.699737:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.699753:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.699771:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.699774:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845038.699778:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.699784:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287adf1a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.702214:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.702335:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641681408 00000100:00100000:3.0:1788845038.702346:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641681408 00000100:00100000:3.0:1788845038.702407:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 912 00000100:00100000:3.0:1788845038.702416:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641681408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.702435:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.702455:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95bc expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.702487:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.702499:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 8) 00010000:00010000:3.0:1788845038.702504:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(14 8) 00010000:00010000:3.0:1788845038.702509:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 14 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.702541:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.702558:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 13 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.702571:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beaad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660332032:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845038.702663:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.706450:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.706540:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641681536 00000100:00100000:1.0:1788845038.706549:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641681536 00002000:00100000:1.0:1788845038.706599:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1069056->1101823) req@ffff94a03beaaa00 x1875739641681536/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.706611:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beaaa00 with x1875739641681536 ext(1069056->1101823) 00010000:00010000:1.0:1788845038.706620:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 14 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.706653:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 14 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845038.706661:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03beaaa00 00000100:00100000:1.0:1788845038.706698:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 24 00000100:00100000:1.0:1788845038.706708:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641681536:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845038.707318:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beaad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660332032:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845038.707348:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 14 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.707364:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:2.0:1788845038.707411:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641681408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4995us (5206us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.707420:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 912 00000100:00100000:1.0:1788845038.712011:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641681536:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5293us (5560us total) trans 4294967452 rc 0/0 00000100:00100000:1.0:1788845038.712030:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 24 00002000:00100000:1.0:1788845038.712045:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1069056->1101823) req@ffff94a03beaaa00 x1875739641681536/t4294967452(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/448 e 0 to 0 dl 1788845049 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.712056:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beaaa00 with x1875739641681536 ext(1069056->1101823) 00010000:00010000:1.0:1788845038.712067:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 16 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.712093:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 16 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845038.712101:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03beaaa00 00000100:00100000:3.0:1788845038.714944:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.715024:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641681664 00000100:00100000:2.0:1788845038.715035:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641681664 00010000:00010000:2.0:1788845038.715074:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 16 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.715108:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 450 00000100:00100000:2.0:1788845038.715118:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641681664:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845038.715132:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845038.715156:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 16 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.715175:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 16 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.715200:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287adea3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1069056->1101823) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9545 expref: 16 pid: 6682 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.715217:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(14 8) 00010000:00010000:2.0:1788845038.715221:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(14 8) 00010000:00010000:2.0:1788845038.715230:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.715240:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.715301:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 13 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.715315:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660332160:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845038.715397:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.719353:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.719394:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.719501:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660332160:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845038.719527:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845038.719581:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641681664:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4464us (4644us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845038.719593:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 450 00000100:00100000:0.0:1788845038.719685:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641681792 00000100:00100000:0.0:1788845038.719691:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641681792 00002000:00010000:0.0:1788845038.719726:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845038.719731:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5500 with x1875739641681792 00000100:00100000:0.0:1788845038.719772:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 872 00000100:00100000:0.0:1788845038.719781:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+22:2369:x1875739641681792:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845038.720150:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 786432 -> 433178 00000004:00010000:0.0:1788845038.720154:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00000004:00010000:0.0:1788845038.720156:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:0.0:1788845038.720158:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00000004:00010000:0.0:1788845038.720161:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845038.720197:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641681792:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 417us (803us total) trans 4294967763 rc 0/0 00000100:00100000:0.0:1788845038.720204:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 872 00002000:00010000:0.0:1788845038.720207:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845038.720209:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5500 with x1875739641681792 00000100:00100000:1.0:1788845038.720740:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641681920 00000100:00100000:1.0:1788845038.720754:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641681920 00002000:00010000:1.0:1788845038.720777:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845038.720782:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc5880 with x1875739641681920 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845038.720789:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc5880. 00000100:00100000:1.0:1788845038.720818:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 444 00000100:00100000:1.0:1788845038.720826:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641681920:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845038.721081:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1199403 -> 0 00002000:00010000:1.0:1788845038.721085:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00002000:00010000:1.0:1788845038.721087:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845038 -> 1788845037 00002000:00010000:1.0:1788845038.721089:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00002000:00010000:1.0:1788845038.721091:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:1.0:1788845038.721134:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641681920:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 309us (1791us total) trans 4294967453 rc 0/0 00000100:00100000:1.0:1788845038.721142:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 444 00002000:00010000:1.0:1788845038.721145:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845038.721148:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc5880 with x1875739641681920 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845038.721152:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc5880. 00000100:00100000:3.0:1788845038.725662:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.725783:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641682048 00000100:00100000:0.0:1788845038.725795:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641682048 00000100:00100000:0.0:1788845038.725864:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1292 00000100:00100000:0.0:1788845038.725875:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641682048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.725919:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.725940:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287adf28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95c3 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.726030:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287adf2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.726045:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287adf2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.726057:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287adf2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845038.726086:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 433178 00000004:00200000:0.0:1788845038.726091:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845038.726110:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287adf2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.726135:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287adf28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95c3 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.726150:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287adf2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95c3 expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.726158:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.726162:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845038.726165:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007800, rc 0) 00000100:00100000:0.0:1788845038.726228:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641682048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 356us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.726236:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1292 00000100:00100000:2.0:1788845038.731020:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.731127:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641682176 00000100:00100000:0.0:1788845038.731138:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641682176 00000100:00100000:0.0:1788845038.731240:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1293 00000100:00100000:0.0:1788845038.731252:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641682176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.731284:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.731305:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adf36 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95ca expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.731392:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.731407:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.731422:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845038.731450:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 433178 00000004:00200000:0.0:1788845038.731454:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845038.731473:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.731498:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adf36 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.731515:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.731524:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.731526:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845038.731529:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845038.731595:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641682176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 345us (716us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.731602:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1293 00000100:00100000:3.0:1788845038.735517:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.735568:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.735652:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641682432 00000100:00100000:0.0:1788845038.735662:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641682432 00000100:00100000:0.0:1788845038.735716:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 913 00000100:00100000:0.0:1788845038.735726:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641682432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.735754:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.735772:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287adf44 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95d8 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.735836:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660332288:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.735946:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.736063:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641682304 00000100:00100000:2.0:1788845038.736069:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641682304 00000100:00100000:2.0:1788845038.736108:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1294 00000100:00100000:2.0:1788845038.736116:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641682304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.736127:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.736141:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adf4b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95d1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.736170:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.736194:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660332416:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.736221:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845038.739429:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845038 00000100:00100000:2.0:1788845038.739466:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660332416:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845038.739491:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 433178 00010000:00010000:2.0:1788845038.739512:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adf4b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95d1 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.739534:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.739537:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845038.739543:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287adf4b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95d1 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.739558:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845038.739629:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641682304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3504us (4114us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.739637:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1294 00002000:00010000:0.0:1788845038.742080:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845037 -> 1788845038 00000100:00100000:0.0:1788845038.742108:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660332288:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.742142:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287adf44 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95d8 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.742166:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287adf44 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95d8 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.742177:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845038.742237:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641682432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6512us (6667us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.742246:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 913 00000100:00100000:3.0:1788845038.745934:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.745994:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.746083:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641682560 00000100:00100000:1.0:1788845038.746093:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641682560 00000100:00100000:1.0:1788845038.746151:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1295 00000100:00100000:1.0:1788845038.746161:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641682560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.746182:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.746203:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287adf52 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95df expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.746239:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.746295:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660332544:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.746375:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.746864:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641682688 00000100:00100000:0.0:1788845038.746871:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641682688 00000100:00100000:0.0:1788845038.746925:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 914 00000100:00100000:0.0:1788845038.746933:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641682688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.746943:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.746955:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287adf59 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95e6 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.746993:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660332672:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.747030:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.750605:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660332672:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.750655:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287adf59 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95e6 expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.750680:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287adf59 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95e6 expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.750693:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845038.750756:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641682688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3823us (4761us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.750764:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 914 00000100:00100000:1.0:1788845038.751337:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660332544:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845038.751357:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 433178 00010000:00010000:1.0:1788845038.751371:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287adf52 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95df expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.751388:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.751392:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845038.751397:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287adf52 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95df expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.751414:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:1.0:1788845038.751465:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641682560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5304us (5541us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.751472:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1295 00000100:00100000:3.0:1788845038.758819:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.759188:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641682816 00000100:00100000:0.0:1788845038.759199:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641682816 00002000:00100000:0.0:1788845038.759253:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035929c00 x1875739641682816/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.759410:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035929c00 with x1875739641682816 00000100:00100000:0.0:1788845038.759500:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 873 00000100:00100000:0.0:1788845038.759510:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641682816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845038.764381:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641682816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4861us (5562us total) trans 0 rc 65536/65536 00000100:00100000:1.0:1788845038.764396:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 873 00002000:00100000:1.0:1788845038.764412:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035929c00 x1875739641682816/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845038.764422:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035929c00 with x1875739641682816 00000100:00100000:3.0:1788845038.770300:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.770370:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641682944 00000100:00100000:0.0:1788845038.770381:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641682944 00002000:00100000:0.0:1788845038.770433:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035929880 x1875739641682944/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.770444:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035929880 with x1875739641682944 00000100:00100000:0.0:1788845038.770482:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 874 00000100:00100000:0.0:1788845038.770492:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641682944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845038.780347:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641682944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 9844us (10046us total) trans 0 rc 65536/65536 00000100:00100000:0.0:1788845038.780362:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 874 00002000:00100000:0.0:1788845038.780376:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035929880 x1875739641682944/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.780387:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035929880 with x1875739641682944 00000100:00100000:2.0:1788845038.783674:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.783781:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.783869:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641683200 00000100:00100000:1.0:1788845038.783878:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641683200 00000100:00100000:1.0:1788845038.783963:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 915 00000100:00100000:1.0:1788845038.783973:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641683200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.783991:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.784010:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adf60 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95f4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.784072:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660332800:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.784156:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.784806:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641683072 00000100:00100000:0.0:1788845038.784812:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641683072 00000100:00100000:0.0:1788845038.784905:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1296 00000100:00100000:0.0:1788845038.784914:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641683072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.784928:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.784942:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adf67 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95ed expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.784970:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.784995:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660332928:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.785044:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.790364:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660332928:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.790394:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 433178 00010000:00010000:0.0:1788845038.790416:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adf67 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ed expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.790437:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.790441:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.790446:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adf67 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ed expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.790460:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845038.790528:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641683072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5614us (6863us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.790536:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1296 00000100:00100000:0.0:1788845038.793342:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660332800:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.793381:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adf60 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95f4 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.793406:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287adf60 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95f4 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.793419:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:0.0:1788845038.793780:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641683200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9807us (9996us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.793789:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 915 00000100:00100000:2.0:1788845038.801504:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.801589:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.801725:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641683456 00000100:00100000:3.0:1788845038.801738:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641683456 00000100:00100000:3.0:1788845038.801812:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 916 00000100:00100000:3.0:1788845038.801824:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641683456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.801846:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.801863:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287adf6e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9602 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.801942:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035928380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660333056:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845038.802030:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.803000:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641683328 00000100:00100000:0.0:1788845038.803009:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641683328 00000100:00100000:0.0:1788845038.803050:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1297 00000100:00100000:0.0:1788845038.803059:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641683328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.803078:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.803099:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287adf75 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a95fb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.803138:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.803171:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660333184:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.803227:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.806525:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660333184:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.806553:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 433178 00010000:00010000:0.0:1788845038.806572:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287adf75 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95fb expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.806593:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.806597:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.806602:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287adf75 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95fb expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.806616:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845038.806690:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641683328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3631us (5440us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.806699:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1297 00000100:00100000:2.0:1788845038.808564:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035928380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660333056:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845038.808602:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287adf6e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9602 expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.808689:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287adf6e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9602 expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.808710:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:2.0:1788845038.808830:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641683456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7007us (7238us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.808839:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 916 00000100:00100000:2.0:1788845038.813053:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.813176:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.813237:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641683712 00000100:00100000:0.0:1788845038.813247:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641683712 00000100:00100000:0.0:1788845038.813328:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 917 00000100:00100000:0.0:1788845038.813339:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641683712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.813358:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.813378:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287adf7c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9610 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.813445:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660333312:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.813532:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.816419:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660333312:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.816462:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287adf7c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9610 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.816487:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287adf7c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9610 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.816501:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845038.816570:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641683712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3232us (3393us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.816579:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 917 00000100:00100000:1.0:1788845038.817599:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641683584 00000100:00100000:1.0:1788845038.817609:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641683584 00000100:00100000:1.0:1788845038.817674:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1298 00000100:00100000:1.0:1788845038.817684:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641683584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.817704:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.817723:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adf83 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9609 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.817773:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.817815:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660333440:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.818007:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.821677:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660333440:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845038.821708:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 433178 00010000:00010000:1.0:1788845038.821728:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adf83 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9609 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.821756:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.821760:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845038.821765:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adf83 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9609 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.821780:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845038.821841:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641683584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4157us (8798us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.821850:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1298 00000100:00100000:3.0:1788845038.825392:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.825617:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641683840 00000100:00100000:1.0:1788845038.825638:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641683840 00000100:00100000:1.0:1788845038.825690:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1299 00000100:00100000:1.0:1788845038.825700:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641683840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.825722:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.825751:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adf8a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9617 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.825786:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.825826:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660333568:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.825942:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.829285:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.829451:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660333568:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845038.829476:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 433178 00010000:00010000:3.0:1788845038.829495:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adf8a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9617 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.829517:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845038.829520:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845038.829525:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287adf8a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9617 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.829538:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:3.0:1788845038.829598:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641683840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3899us (4216us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.829607:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1299 00000100:00100000:1.0:1788845038.829751:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641683968 00000100:00100000:1.0:1788845038.829758:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641683968 00000100:00100000:1.0:1788845038.829809:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 918 00000100:00100000:1.0:1788845038.829818:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641683968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.829833:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.829848:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adf91 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a961e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.829941:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660333696:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.829999:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.832566:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660333696:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845038.832603:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adf91 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a961e expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.832640:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287adf91 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a961e expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.832660:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845038.832721:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641683968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2901us (3455us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845038.832731:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 918 00000100:00100000:2.0:1788845038.837319:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.837361:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.837508:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641684096 00000100:00100000:0.0:1788845038.837518:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641684096 00000100:00100000:0.0:1788845038.837580:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1300 00000100:00100000:0.0:1788845038.837589:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641684096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.837610:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.837640:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adf98 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9625 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.837682:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.837728:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660333824:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.837824:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.837940:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641684224 00000100:00100000:3.0:1788845038.837945:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641684224 00000100:00100000:3.0:1788845038.837984:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 919 00000100:00100000:3.0:1788845038.837992:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641684224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.838001:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.838013:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287adf9f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a962c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.838048:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660333952:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845038.838079:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.841568:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660333824:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.841595:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 433178 00010000:00010000:0.0:1788845038.841615:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adf98 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9625 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.841649:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.841651:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.841656:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adf98 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9625 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.841671:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845038.841735:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641684096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4146us (4425us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.841754:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1300 00000100:00100000:3.0:1788845038.841864:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660333952:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845038.841933:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287adf9f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a962c expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.841950:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287adf9f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a962c expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.841961:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845038.841985:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641684224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3994us (4623us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.841991:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 919 00000100:00100000:2.0:1788845038.851123:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.851228:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641684352 00000100:00100000:0.0:1788845038.851237:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641684352 00000100:00100000:0.0:1788845038.851304:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1301 00000100:00100000:0.0:1788845038.851315:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641684352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.851336:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.851356:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adfa6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9633 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.851395:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.851435:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660334080:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.851518:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845038.853272:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.853415:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641684480 00000100:00100000:3.0:1788845038.853426:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641684480 00000100:00100000:3.0:1788845038.853486:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 920 00000100:00100000:3.0:1788845038.853496:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641684480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.853516:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.853534:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287adfad lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a963a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.853590:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660334208:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845038.853677:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.857859:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660334208:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845038.857951:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287adfad lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a963a expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.857976:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287adfad lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a963a expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.857989:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845038.858052:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641684480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4557us (4797us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.858061:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 920 00000100:00100000:0.0:1788845038.858680:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660334080:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845038.858697:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 433178 00010000:00010000:0.0:1788845038.858710:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adfa6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9633 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.858725:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.858728:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.858732:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adfa6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9633 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.858757:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845038.858799:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641684352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7485us (7687us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.858806:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1301 00000100:00100000:2.0:1788845038.864129:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.864215:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641684736 00000100:00100000:1.0:1788845038.864225:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641684736 00000100:00100000:1.0:1788845038.864300:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1302 00000100:00100000:1.0:1788845038.864310:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641684736:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845038.864369:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845038.864398:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287adf2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95c3 expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.864417:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.864422:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845038.864637:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.864652:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.864660:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.864695:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.864719:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 11 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.864733:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660334336:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845038.864823:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.867584:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660334336:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845038.867612:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.869219:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.869752:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641684864 00000100:00100000:1.0:1788845038.869761:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641684864 00010000:00010000:1.0:1788845038.869799:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 12 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.869841:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 451 00000100:00100000:1.0:1788845038.869850:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641684864:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845038.869865:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845038.869871:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 12 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.869908:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 12 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.869931:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287adf3d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95ca expref: 12 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.869954:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.869959:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.869969:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.869976:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.870062:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845038.870117:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641684864:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 267us (904us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.870124:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 451 00010000:00010000:2.0:1788845038.874044:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.874062:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845038.874200:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967764 00000004:00010000:2.0:1788845038.875028:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 433178 -> 433178 00000004:00010000:2.0:1788845038.875034:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00000004:00010000:2.0:1788845038.875037:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845038 -> 1788845036 00000004:00010000:2.0:1788845038.875040:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00000004:00010000:2.0:1788845038.875042:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:2.0:1788845038.875053:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000564716e3 reply state 000000007660aa7d transno 4294967765 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.875066:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845038.875415:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+26:14891:x1875739641684736:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 11101us (11290us total) trans 4294967765 rc 0/0 00000100:00100000:2.0:1788845038.875423:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1302 00000020:00080000:2.0:1788845038.875428:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845038.880370:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.880415:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.880442:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641685120 00000100:00100000:3.0:1788845038.880452:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641685120 00002000:00010000:3.0:1788845038.880495:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845038.880499:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035928380 with x1875739641685120 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845038.880512:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035928380. 00000100:00100000:3.0:1788845038.880538:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 445 00000100:00100000:3.0:1788845038.880549:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641685120:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845038.880792:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1788845038.880797:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00002000:00010000:3.0:1788845038.880799:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845038 -> 1788845037 00002000:00010000:3.0:1788845038.880801:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00002000:00010000:3.0:1788845038.880803:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:1.0:1788845038.880908:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641684992 00000100:00100000:1.0:1788845038.880913:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641684992 00002000:00010000:1.0:1788845038.880934:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845038.880938:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03592b480 with x1875739641684992 00000100:00100000:1.0:1788845038.880963:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 875 00000100:00100000:1.0:1788845038.880970:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2370:x1875739641684992:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845038.881093:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 433178 -> 602308 00000004:00010000:1.0:1788845038.881097:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845038 -> 1788845038 00000004:00010000:1.0:1788845038.881099:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845038.881101:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845038 -> 1788845038 00000004:00010000:1.0:1788845038.881104:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00010000:00010000:0.0:1788845038.881190:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.881200:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.881219:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.881225:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845038.881229:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.881237:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.881253:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.881256:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845038.881261:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.881267:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287adfb4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.888477:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641685120:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 7918us (8049us total) trans 4294967454 rc 0/0 00000100:00100000:3.0:1788845038.888493:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 445 00002000:00010000:3.0:1788845038.888501:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845038.888504:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035928380 with x1875739641685120 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845038.888516:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035928380. 00000100:00100000:1.0:1788845038.888583:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2370:x1875739641684992:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 7612us (8223us total) trans 4294967766 rc 0/0 00000100:00100000:1.0:1788845038.888593:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 875 00002000:00010000:1.0:1788845038.888597:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845038.888601:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03592b480 with x1875739641684992 00000100:00100000:0.0:1788845038.894563:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.895252:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641685248 00000100:00100000:3.0:1788845038.895397:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641685248 00000100:00100000:0.0:1788845038.895595:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1303 00000100:00100000:0.0:1788845038.895607:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:14891:x1875739641685248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.895642:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.895662:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adfbb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9641 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.895762:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287adfc2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.895776:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287adfc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.895787:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287adfc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845038.895813:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 602308 00000004:00200000:0.0:1788845038.895817:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845038.895835:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287adfc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.895861:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287adfbb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9641 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.895877:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287adfc2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9641 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.895909:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.895912:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845038.895916:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845038.895977:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641685248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 372us (1424us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.895984:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1303 00000100:00100000:1.0:1788845038.900401:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.900576:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641685376 00000100:00100000:3.0:1788845038.900588:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641685376 00002000:00100000:3.0:1788845038.900658:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964700 x1875739641685376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845038.900670:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964700 with x1875739641685376 00000100:00100000:3.0:1788845038.900709:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 876 00000100:00100000:3.0:1788845038.900720:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641685376:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845038.904149:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641685376:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 3420us (3748us total) trans 0 rc 65536/65536 00000100:00100000:3.0:1788845038.904163:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 876 00002000:00100000:3.0:1788845038.904176:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964700 x1875739641685376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845038.904187:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964700 with x1875739641685376 00000100:00100000:3.0:1788845038.907207:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.907344:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641685504 00000100:00100000:1.0:1788845038.907356:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641685504 00000100:00100000:1.0:1788845038.907427:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1304 00000100:00100000:1.0:1788845038.907438:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641685504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.907462:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.907486:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adfc9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9648 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.907581:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.907596:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.907608:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845038.907647:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 602308 00000004:00200000:1.0:1788845038.907651:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845038.907672:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.907947:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287adfc9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.907970:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.907978:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.907980:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845038.907983:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:1.0:1788845038.908044:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641685504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 608us (851us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845038.908051:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1304 00000100:00100000:1.0:1788845038.911207:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.911297:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641685632 00000100:00100000:2.0:1788845038.911308:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641685632 00000100:00100000:2.0:1788845038.911368:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1305 00000100:00100000:2.0:1788845038.911379:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641685632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.911403:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.911424:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287adfd7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a964f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.911460:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.911506:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660334464:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.911587:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845038.914533:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.914607:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641685760 00000100:00100000:0.0:1788845038.914618:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641685760 00000100:00100000:0.0:1788845038.914676:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 921 00000100:00100000:0.0:1788845038.914686:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641685760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.914705:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.914722:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287adfde lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9656 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.914790:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660334592:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.914868:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845038.917026:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845038 00000100:00100000:2.0:1788845038.917059:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660334464:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845038.917084:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602308 00010000:00010000:2.0:1788845038.917102:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287adfd7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a964f expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.917121:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845038.917125:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845038.917130:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287adfd7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a964f expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845038.917145:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845038.917205:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641685632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5827us (6009us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845038.917214:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1305 00002000:00010000:0.0:1788845038.920935:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845037 -> 1788845038 00000100:00100000:0.0:1788845038.920971:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660334592:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.921017:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287adfde lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9656 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.921043:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287adfde lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9656 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.921056:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845038.921117:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641685760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6432us (6593us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.921126:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 921 00000100:00100000:1.0:1788845038.924790:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845038.925239:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641685888 00000100:00100000:3.0:1788845038.925250:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641685888 00000100:00100000:3.0:1788845038.925324:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1306 00000100:00100000:3.0:1788845038.925335:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641685888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845038.925356:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845038.925378:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adfe5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a965d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.925413:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845038.925452:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660334720:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845038.925536:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.927805:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.927914:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641686016 00000100:00100000:1.0:1788845038.927923:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641686016 00000100:00100000:1.0:1788845038.927972:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 922 00000100:00100000:1.0:1788845038.927981:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641686016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845038.927998:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845038.928013:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287adfec lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9664 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.928070:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660334848:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845038.928148:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.931355:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660334720:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845038.931384:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602308 00010000:00010000:3.0:1788845038.931402:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adfe5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a965d expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.931423:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845038.931426:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845038.931431:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287adfe5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a965d expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.931448:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845038.931520:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641685888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6186us (6739us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.931533:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1306 00000100:00100000:0.0:1788845038.931850:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660334848:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.931875:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287adfec lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9664 expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.931926:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287adfec lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9664 expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.931938:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:0.0:1788845038.931981:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641686016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4000us (4182us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.931988:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 922 00000100:00100000:1.0:1788845038.940335:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.940768:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641686144 00000100:00100000:0.0:1788845038.940777:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641686144 00000100:00100000:0.0:1788845038.940922:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1307 00000100:00100000:0.0:1788845038.940933:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641686144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.940956:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.940976:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a966b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.941005:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.941015:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.941047:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.941064:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 26 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.941077:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660334976:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845038.941160:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.943915:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660334976:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845038.943954:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.943975:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.943978:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.943982:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.943996:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845038.944047:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641686144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3116us (3722us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.944056:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1307 00000100:00100000:3.0:1788845038.947376:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845038.947546:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641686272 00000100:00100000:1.0:1788845038.947557:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641686272 00010000:00010000:1.0:1788845038.947598:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 27 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.947651:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 452 00000100:00100000:1.0:1788845038.947661:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641686272:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845038.947675:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845038.947702:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 27 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.947720:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 27 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.947752:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287adecd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a956f expref: 27 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.947770:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845038.947776:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.947786:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.947794:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845038.947833:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845038.947851:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660335104:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845038.947978:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.953866:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660335104:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845038.953922:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845038.953983:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641686272:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6323us (6615us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.953993:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 452 00000100:00100000:3.0:1788845038.955712:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.955976:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641686400 00000100:00100000:0.0:1788845038.955986:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641686400 00002000:00100000:0.0:1788845038.956035:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687b80 x1875739641686400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/0 e 0 to 0 dl 1788845049 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.956047:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687b80 with x1875739641686400 00000100:00100000:0.0:1788845038.956084:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 877 00000100:00100000:0.0:1788845038.956094:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641686400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845038.966729:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641686400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 10623us (11023us total) trans 0 rc 606208/606208 00000100:00100000:0.0:1788845038.966758:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 877 00002000:00100000:0.0:1788845038.966771:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035687b80 x1875739641686400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:144/0 lens 488/440 e 0 to 0 dl 1788845049 ref 1 fl Complete:/600/0 rc 606208/606208 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845038.966782:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035687b80 with x1875739641686400 00000100:00100000:3.0:1788845038.970768:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.970918:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641686528 00000100:00100000:2.0:1788845038.970929:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641686528 00000100:00100000:2.0:1788845038.970980:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 923 00000100:00100000:2.0:1788845038.970990:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641686528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.971008:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.971028:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287adffa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9672 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.971090:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660335232:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.971179:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.973962:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660335232:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845038.974006:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287adffa lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9672 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.974034:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287adffa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9672 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.974049:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:3.0:1788845038.974116:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641686528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3126us (3356us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.974125:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 923 00000100:00100000:3.0:1788845038.977692:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845038.977847:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641686656 00000100:00100000:2.0:1788845038.977857:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641686656 00000100:00100000:2.0:1788845038.977949:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 924 00000100:00100000:2.0:1788845038.977959:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641686656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845038.977978:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845038.977996:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ae001 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9679 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845038.978058:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660335360:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845038.978142:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845038.981313:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660335360:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845038.981356:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ae001 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9679 expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.981384:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ae001 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9679 expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845038.981397:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033a00, rc 0) 00000100:00100000:3.0:1788845038.981465:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641686656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3507us (3782us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845038.981475:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 924 00000100:00100000:1.0:1788845038.985362:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.985466:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641686784 00000100:00100000:0.0:1788845038.985476:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641686784 00000100:00100000:0.0:1788845038.985535:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 925 00000100:00100000:0.0:1788845038.985546:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641686784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.985565:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.985585:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae008 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9680 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.985656:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660335488:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.985754:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.988647:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660335488:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.988689:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae008 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9680 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.988714:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae008 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9680 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.988726:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845038.988797:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641686784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3252us (3443us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.988806:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 925 00000100:00100000:2.0:1788845038.992010:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.992122:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641686912 00000100:00100000:0.0:1788845038.992132:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641686912 00000100:00100000:0.0:1788845038.992190:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 926 00000100:00100000:0.0:1788845038.992200:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641686912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.992219:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.992236:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae00f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9687 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845038.992307:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660335616:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845038.992382:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845038.995089:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660335616:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845038.995129:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae00f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9687 expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.995156:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae00f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9687 expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.995172:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:0.0:1788845038.995238:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641686912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3038us (3235us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845038.995247:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 926 00000100:00100000:1.0:1788845038.999054:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845038.999136:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641687040 00000100:00100000:0.0:1788845038.999145:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641687040 00000100:00100000:0.0:1788845038.999206:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1308 00000100:00100000:0.0:1788845038.999217:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641687040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845038.999238:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845038.999259:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae016 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a968e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.999306:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae016 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a968e expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845038.999324:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602308 00010000:00010000:0.0:1788845038.999332:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae016 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a968e expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845038.999346:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845038.999349:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845038.999352:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845038.999411:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641687040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 196us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845038.999418:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1308 00000100:00100000:2.0:1788845039.003676:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.003794:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641687168 00000100:00100000:3.0:1788845039.003806:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641687168 00000100:00100000:3.0:1788845039.003924:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 927 00000100:00100000:3.0:1788845039.003938:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641687168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.003956:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.003974:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae01d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9695 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.004036:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660335744:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.004112:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00080000:0.0:1788845039.004180:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845039.006655:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660335744:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.006706:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae01d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9695 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.006731:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae01d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9695 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.006753:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:2.0:1788845039.006819:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641687168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2884us (3154us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.006827:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 927 00000100:00100000:1.0:1788845039.013025:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.013520:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641687296 00000100:00100000:3.0:1788845039.013532:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641687296 00000100:00100000:3.0:1788845039.013594:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 928 00000100:00100000:3.0:1788845039.013605:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641687296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.013633:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.013654:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae024 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a969c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.013714:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660335872:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.013995:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.016797:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660335872:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845039.016837:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae024 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a969c expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.016869:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae024 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a969c expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.016900:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845039.017001:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641687296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3397us (3986us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.017010:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 928 00000100:00100000:1.0:1788845039.020763:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.020878:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641687424 00000100:00100000:2.0:1788845039.020990:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641687424 00000100:00100000:2.0:1788845039.021078:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 929 00000100:00100000:2.0:1788845039.021089:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641687424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.021109:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.021130:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae02b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96a3 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.021191:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660336000:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.021428:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.024185:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660336000:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.024225:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae02b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96a3 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.024251:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae02b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96a3 expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.024382:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:2.0:1788845039.024477:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641687424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3388us (3723us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.024486:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 929 00000100:00100000:2.0:1788845039.027657:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.027794:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641687552 00000100:00100000:1.0:1788845039.027804:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641687552 00000100:00100000:1.0:1788845039.027868:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 930 00000100:00100000:1.0:1788845039.027878:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641687552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.027924:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.027941:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ae032 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96aa expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.028003:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02482a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660336128:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.028084:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.030925:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02482a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660336128:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.030963:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ae032 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96aa expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.030987:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ae032 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96aa expref: 16 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.031000:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:2.0:1788845039.031052:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641687552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3175us (3404us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.031060:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 930 00000100:00100000:2.0:1788845039.034207:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.034286:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641687680 00000100:00100000:3.0:1788845039.034296:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641687680 00000100:00100000:3.0:1788845039.034355:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 931 00000100:00100000:3.0:1788845039.034365:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641687680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.034385:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.034402:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae039 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96b1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.034460:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660336256:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.034541:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.037835:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660336256:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.037877:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae039 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b1 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.037986:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae039 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b1 expref: 16 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.038000:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:2.0:1788845039.038114:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641687680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3749us (3915us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.038122:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 931 00000100:00100000:1.0:1788845039.044065:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.044147:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641687936 00000100:00100000:2.0:1788845039.044158:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641687936 00000100:00100000:2.0:1788845039.044240:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1309 00000100:00100000:2.0:1788845039.044250:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641687936:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845039.044321:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845039.044351:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287adfc2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9641 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.044373:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.044378:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845039.044606:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.044635:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.044645:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.044683:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.044702:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 12 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.044715:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660336384:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845039.044815:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fcc0 going to sleep for 11 seconds 00000100:00080000:0.0:1788845039.045227:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845039.047957:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660336384:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845039.047997:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.049607:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.049807:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641688064 00000100:00100000:2.0:1788845039.049816:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641688064 00010000:00010000:2.0:1788845039.049876:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 13 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.049960:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 453 00000100:00100000:2.0:1788845039.049971:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019e6cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641688064:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845039.049990:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845039.049998:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 13 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.050018:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 13 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.050048:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287adfd0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9648 expref: 13 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.050064:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.050069:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.050080:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.050087:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.050113:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845039.050173:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019e6cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641688064:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 201us (574us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.050181:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 453 00000100:00080000:0.0:1788845039.050296:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788845039.050593:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.050602:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845039.050751:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967767 00000004:00010000:1.0:1788845039.050804:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 602308 -> 602308 00000004:00010000:1.0:1788845039.050809:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845038 -> 1788845039 00000004:00010000:1.0:1788845039.050811:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845038 -> 1788845036 00000004:00010000:1.0:1788845039.050814:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845038 -> 1788845039 00000004:00010000:1.0:1788845039.050816:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:1.0:1788845039.050823:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000108cd807 reply state 00000000a6aa9da5 transno 4294967768 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.050832:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845039.050943:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:14891:x1875739641687936:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6694us (6888us total) trans 4294967768 rc 0/0 00000100:00100000:1.0:1788845039.050950:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1309 00000020:00080000:1.0:1788845039.050954:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845039.054860:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.054877:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.054923:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.054929:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02032000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845039.054934:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.054941:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.054964:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.054967:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845039.054971:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.054977:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae040 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.055596:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.055794:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641688192 00000100:00100000:0.0:1788845039.055804:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641688192 00000100:00100000:0.0:1788845039.055857:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1310 00000100:00100000:0.0:1788845039.055867:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641688192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.055934:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.055937:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788845039.055975:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae016 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a968e expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.055988:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.055991:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845039.056006:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96b8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.056031:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.056041:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.056081:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.056100:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 11 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.056114:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660336512:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845039.056190:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.059964:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660336512:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845039.060004:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.060023:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.060027:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.060031:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.060042:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:0.0:1788845039.060095:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641688192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4229us (4502us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.060105:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1310 00000100:00100000:1.0:1788845039.064100:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.064233:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641688320 00000100:00100000:2.0:1788845039.064243:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641688320 00010000:00010000:2.0:1788845039.064305:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 12 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.064342:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 454 00000100:00100000:2.0:1788845039.064353:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641688320:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845039.064367:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845039.064395:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 12 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.064417:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 12 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.064439:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287adff3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a966b expref: 12 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.064455:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.064461:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.064471:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.064480:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.064522:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.064537:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660336640:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845039.064634:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.069549:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660336640:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845039.069577:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845039.069639:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641688320:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5287us (5548us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.069653:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 454 00000100:00100000:3.0:1788845039.070223:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.070712:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641688448 00000100:00100000:1.0:1788845039.070721:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641688448 00002000:00010000:1.0:1788845039.070786:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1875067->18446744073709551615) 00002000:00100000:1.0:1788845039.070791:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329500 with x1875739641688448 ext(1875067->18446744073709551615) 00002000:00010000:1.0:1788845039.070803:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329500. 00000100:00100000:1.0:1788845039.070834:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 446 00000100:00100000:1.0:1788845039.070845:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641688448:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845039.071111:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1875067 00002000:00010000:1.0:1788845039.071114:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845038 -> 1788845039 00002000:00010000:1.0:1788845039.071117:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845038 -> 1788845037 00002000:00010000:1.0:1788845039.071119:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845038 -> 1788845039 00002000:00010000:1.0:1788845039.071121:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:1.0:1788845039.071171:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641688448:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 328us (948us total) trans 4294967455 rc 0/0 00000100:00100000:1.0:1788845039.071179:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 446 00002000:00010000:1.0:1788845039.071183:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1875067->18446744073709551615) 00002000:00100000:1.0:1788845039.071185:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329500 with x1875739641688448 ext(1875067->18446744073709551615) 00002000:00010000:1.0:1788845039.071191:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329500. 00000100:00080000:2.0:1788845039.071661:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845039.075336:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.075450:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641688576 00000100:00100000:0.0:1788845039.075461:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641688576 00000100:00100000:0.0:1788845039.075524:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1311 00000100:00100000:0.0:1788845039.075534:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641688576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.075557:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.075577:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae04e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96bf expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.075676:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.075691:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.075703:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845039.075732:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1875067 00000004:00200000:0.0:1788845039.075737:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845039.075774:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.075801:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae04e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.075819:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.075828:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.075830:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845039.075834:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845039.075926:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641688576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 393us (600us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.075932:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1311 00000100:00100000:1.0:1788845039.079871:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.080058:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641688704 00000100:00100000:3.0:1788845039.080070:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641688704 00000100:00100000:3.0:1788845039.080137:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1312 00000100:00100000:3.0:1788845039.080150:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641688704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.080174:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.080198:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae05c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96c6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.080239:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.080447:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036083480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660336768:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.080528:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.083039:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.083153:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641688832 00000100:00100000:2.0:1788845039.083162:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641688832 00000100:00100000:2.0:1788845039.083219:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 932 00000100:00100000:2.0:1788845039.083229:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641688832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.083247:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.083278:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae063 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96cd expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.083337:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660336896:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.083424:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788845039.085822:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845038 00000100:00100000:3.0:1788845039.085854:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036083480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660336768:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845039.085878:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602308 00010000:00010000:3.0:1788845039.085940:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae05c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96c6 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.085961:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.085964:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845039.085970:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae05c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96c6 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.085984:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845039.086044:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641688704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5897us (6182us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.086053:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1312 00002000:00010000:2.0:1788845039.088773:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845037 -> 1788845038 00000100:00100000:2.0:1788845039.088806:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660336896:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.088843:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae063 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96cd expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.088870:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae063 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96cd expref: 16 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.088910:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:2.0:1788845039.088973:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641688832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5745us (5945us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.088981:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 932 00000100:00100000:1.0:1788845039.093117:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.093179:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.093283:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641688960 00000100:00100000:0.0:1788845039.093292:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641688960 00000100:00100000:0.0:1788845039.093354:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1313 00000100:00100000:0.0:1788845039.093364:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641688960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.093387:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.093410:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae06a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96d4 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.093452:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.093492:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660337024:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.093575:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.094390:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641689088 00000100:00100000:2.0:1788845039.094396:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641689088 00000100:00100000:2.0:1788845039.094445:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 933 00000100:00100000:2.0:1788845039.094453:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641689088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.094466:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.094480:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae071 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96db expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.094522:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660337152:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.094581:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.098604:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660337152:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.098662:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae071 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96db expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.098688:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae071 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96db expref: 16 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.098704:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845039.098780:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641689088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4327us (5600us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.098790:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 933 00000100:00100000:0.0:1788845039.099567:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660337024:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.099585:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602308 00010000:00010000:0.0:1788845039.099600:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae06a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96d4 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.099628:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.099631:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.099635:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae06a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96d4 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.099646:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845039.099684:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641688960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6322us (6579us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.099691:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1313 00000100:00100000:1.0:1788845039.104278:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.104421:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641689216 00000100:00100000:2.0:1788845039.104431:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641689216 00000100:00100000:2.0:1788845039.104491:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 934 00000100:00100000:2.0:1788845039.104501:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641689216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.104521:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.104540:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96e2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.104573:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 16 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.104583:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(14 8) 00010000:00010000:2.0:1788845039.104588:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 14) 00010000:00010000:2.0:1788845039.104594:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1449984->1478655] (req 1449984->1478655) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 16 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.104641:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 16 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.104660:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 16 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.104675:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660337280:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845039.104779:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.107609:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660337280:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845039.107657:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1449984->1478655] (req 1449984->1478655) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 16 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.107679:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845039.107747:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641689216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3239us (3482us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.107756:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 934 00000100:00100000:1.0:1788845039.111976:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.112139:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641689344 00000100:00100000:0.0:1788845039.112149:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641689344 00002000:00100000:0.0:1788845039.112200:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1830911) req@ffff94a0360b6680 x1875739641689344/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 16872/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.112213:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b6680 with x1875739641689344 ext(1814528->1830911) 00010000:00010000:0.0:1788845039.112223:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 17 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.112244:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 17 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845039.112253:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a0360b6680 00000100:00100000:0.0:1788845039.112303:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 25 00000100:00100000:0.0:1788845039.112314:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641689344:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845039.112801:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641689344:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 486us (833us total) trans 4294967456 rc 0/0 00000100:00100000:0.0:1788845039.112809:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 25 00002000:00100000:0.0:1788845039.112816:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1830911) req@ffff94a0360b6680 x1875739641689344/t4294967456(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 16872/448 e 0 to 0 dl 1788845050 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.112825:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b6680 with x1875739641689344 ext(1814528->1830911) 00010000:00010000:0.0:1788845039.112832:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 19 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.112846:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 19 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845039.112859:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a0360b6680 00000100:00100000:1.0:1788845039.116980:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.117083:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641689472 00000100:00100000:2.0:1788845039.117094:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641689472 00010000:00010000:2.0:1788845039.117135:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 19 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.117180:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:2.0:1788845039.117190:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641689472:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845039.117204:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845039.117228:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 19 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.117245:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 19 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.117496:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287adf21 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a95bc expref: 19 pid: 6685 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.117515:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 14) 00010000:00010000:2.0:1788845039.117520:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 14) 00010000:00010000:2.0:1788845039.117528:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 15 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.117538:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 15 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.117579:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 15 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.117595:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660337408:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845039.117771:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.126449:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660337408:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845039.126479:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845039.126551:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641689472:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9362us (9579us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.126563:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:1.0:1788845039.128535:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.128631:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641689600 00000100:00100000:0.0:1788845039.128640:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641689600 00000100:00100000:0.0:1788845039.128710:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 447 00000100:00100000:0.0:1788845039.128721:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641689600:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:0.0:1788845039.129027:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1875067 -> 1875067 00002000:00010000:0.0:1788845039.129032:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00002000:00010000:0.0:1788845039.129034:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845038 -> 1788845038 00002000:00010000:0.0:1788845039.129036:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00002000:00010000:0.0:1788845039.129038:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:0.0:1788845039.129091:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641689600:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 372us (565us total) trans 4294967457 rc 0/0 00000100:00100000:0.0:1788845039.129099:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 447 00000100:00100000:0.0:1788845039.132445:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.132563:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641689728 00000100:00100000:1.0:1788845039.132573:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641689728 00000100:00100000:1.0:1788845039.132717:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1314 00000100:00100000:1.0:1788845039.132729:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641689728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.132758:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.132781:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae07f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96e9 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.132817:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.132858:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660337536:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.132968:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.135098:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660337536:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.135129:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602308 00010000:00010000:0.0:1788845039.135147:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae07f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e9 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.135168:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.135171:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.135176:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae07f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e9 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.135191:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:0.0:1788845039.135418:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641689728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2691us (2982us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.135426:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1314 00000100:00100000:2.0:1788845039.138973:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.139119:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641689856 00000100:00100000:1.0:1788845039.139129:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641689856 00000100:00100000:1.0:1788845039.139188:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1315 00000100:00100000:1.0:1788845039.139199:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641689856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.139220:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.139241:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae086 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96f0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.139292:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.139333:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660337664:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.139415:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.142016:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660337664:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845039.142043:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602308 00010000:00010000:1.0:1788845039.142060:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae086 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96f0 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.142081:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.142084:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.142088:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae086 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96f0 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.142101:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845039.142161:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641689856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2964us (3196us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.142169:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1315 00000100:00100000:2.0:1788845039.147785:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.150982:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641689984 00000100:00100000:0.0:1788845039.150996:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641689984 00000100:00100000:0.0:1788845039.151110:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 448 00000100:00100000:0.0:1788845039.151120:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641689984:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:0.0:1788845039.151386:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1875067 -> 1875067 00002000:00010000:0.0:1788845039.151390:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00002000:00010000:0.0:1788845039.151393:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845038 -> 1788845038 00002000:00010000:0.0:1788845039.151395:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00002000:00010000:0.0:1788845039.151397:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 317 00000100:00100000:0.0:1788845039.151465:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641689984:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 347us (3690us total) trans 4294967458 rc 0/0 00000100:00100000:0.0:1788845039.151472:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 448 00000100:00100000:2.0:1788845039.154243:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.154757:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641690112 00000100:00100000:3.0:1788845039.154767:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641690112 00000100:00100000:3.0:1788845039.154867:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1316 00000100:00100000:3.0:1788845039.154879:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641690112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.154942:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.154964:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae08d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96f7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.155054:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae094 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.155068:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae094 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.155080:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae094 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845039.155106:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1875067 00000004:00200000:3.0:1788845039.155111:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845039.155130:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae094 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.155156:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae08d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96f7 expref: 30 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.155175:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae094 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96f7 expref: 29 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.155182:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.155185:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845039.155189:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:3.0:1788845039.155251:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+29:14891:x1875739641690112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 375us (1018us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845039.155458:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1316 00000100:00100000:2.0:1788845039.158610:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.158780:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641690240 00000100:00100000:3.0:1788845039.158790:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641690240 00000100:00100000:3.0:1788845039.158905:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 935 00000100:00100000:3.0:1788845039.158916:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641690240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.158935:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.158954:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae09b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a96fe expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.159016:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036083480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660337792:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.159149:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.162452:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036083480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660337792:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845039.162493:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae09b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96fe expref: 19 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.162517:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae09b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96fe expref: 19 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.162531:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:3.0:1788845039.162641:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641690240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3726us (4041us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.162650:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 935 00000100:00100000:1.0:1788845039.166140:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.166236:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641690368 00000100:00100000:3.0:1788845039.166248:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641690368 00000100:00100000:3.0:1788845039.166525:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 936 00000100:00100000:3.0:1788845039.166537:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641690368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.166560:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.166578:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae0a2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9705 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.166646:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036083480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660337920:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.166730:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.169275:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036083480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660337920:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845039.169323:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae0a2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9705 expref: 19 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.169348:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae0a2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9705 expref: 19 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.169363:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:0.0:1788845039.169483:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641690368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2948us (3355us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.169496:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 936 00000100:00100000:1.0:1788845039.178801:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.178917:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641690496 00000100:00100000:2.0:1788845039.178927:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641690496 00000100:00100000:2.0:1788845039.178990:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1317 00000100:00100000:2.0:1788845039.179002:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641690496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.179024:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.179045:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae0a9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a970c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.179080:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.179124:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660338048:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.179198:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845039.181515:0:5840:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 602308 -> 724182 00000100:00100000:2.0:1788845039.181551:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660338048:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845039.181575:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:2.0:1788845039.181595:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae0a9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a970c expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.181626:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.181630:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845039.181636:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae0a9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a970c expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.181651:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033000, rc 0) 00000100:00100000:2.0:1788845039.181713:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641690496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2711us (2921us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.181721:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1317 00000100:00100000:3.0:1788845039.184630:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.184721:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641690624 00000100:00100000:1.0:1788845039.184731:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641690624 00000100:00100000:1.0:1788845039.184797:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1318 00000100:00100000:1.0:1788845039.184807:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641690624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.184828:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.184849:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0b0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9713 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.184928:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.184970:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660338176:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.185045:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786d80 going to sleep for 11 seconds 00010000:00010000:1.0:1788845039.186167:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845039.187450:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660338176:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845039.187490:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:2.0:1788845039.187509:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0b0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9713 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.187529:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.187532:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845039.187537:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0b0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9713 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.187552:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:2.0:1788845039.187622:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641690624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2815us (3011us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.187630:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1318 00000100:00100000:2.0:1788845039.190759:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.190838:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641690752 00000100:00100000:3.0:1788845039.190849:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641690752 00000100:00100000:3.0:1788845039.190944:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 937 00000100:00100000:3.0:1788845039.190955:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641690752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.190979:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.190998:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a971a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.191030:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 17 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.191039:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 14) 00010000:00010000:3.0:1788845039.191044:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 14) 00010000:00010000:3.0:1788845039.191050:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1400832->1449983] (req 1400832->1449983) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 19 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.191083:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 17 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.191107:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 17 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.191122:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a036082680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660338304:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845039.191201:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009d23600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.195057:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.195175:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a036082680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660338304:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845039.195212:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1400832->1449983] (req 1400832->1449983) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 19 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.195233:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:2.0:1788845039.195297:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641690752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4345us (4558us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.195306:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 937 00000100:00100000:1.0:1788845039.195948:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641690880 00000100:00100000:1.0:1788845039.195958:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641690880 00010000:00010000:1.0:1788845039.196001:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 18 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.196037:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 456 00000100:00100000:1.0:1788845039.196047:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641690880:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845039.196063:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845039.196089:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 18 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.196108:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 18 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.196133:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae078 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1449984->1478655) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96e2 expref: 18 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.196148:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 14) 00010000:00010000:1.0:1788845039.196152:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 14) 00010000:00010000:1.0:1788845039.196160:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 18 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.196170:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 18 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.196204:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 18 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.196220:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660338432:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845039.196622:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009786b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.201506:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.201603:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641691008 00000100:00100000:2.0:1788845039.201623:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641691008 00002000:00100000:2.0:1788845039.201676:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1400832->1404927) req@ffff949f01dbad80 x1875739641691008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.201688:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01dbad80 with x1875739641691008 ext(1400832->1404927) 00002000:00010000:2.0:1788845039.201693:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01dbad80 00000100:00100000:2.0:1788845039.201730:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 449 00000100:00100000:2.0:1788845039.201756:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641691008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845039.202048:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbb100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660338432:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845039.202071:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845039.202139:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641690880:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6088us (7086us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.202148:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 456 00000100:00100000:2.0:1788845039.203178:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641691008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1435us (1677us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845039.203193:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 449 00002000:00100000:2.0:1788845039.203204:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1400832->1404927) req@ffff949f01dbad80 x1875739641691008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/4536 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.203214:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01dbad80 with x1875739641691008 ext(1400832->1404927) 00002000:00010000:2.0:1788845039.203219:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01dbad80 00000100:00100000:1.0:1788845039.206835:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.206942:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641691136 00000100:00100000:3.0:1788845039.206955:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641691136 00002000:00100000:3.0:1788845039.207009:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1449983) req@ffff949f01dbb100 x1875739641691136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.207021:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01dbb100 with x1875739641691136 ext(1445888->1449983) 00002000:00010000:3.0:1788845039.207027:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01dbb100 00000100:00100000:3.0:1788845039.207064:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 450 00000100:00100000:3.0:1788845039.207074:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641691136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845039.207433:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641691136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 359us (607us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845039.207442:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 450 00002000:00100000:3.0:1788845039.207449:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1449983) req@ffff949f01dbb100 x1875739641691136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/4536 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.207458:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01dbb100 with x1875739641691136 ext(1445888->1449983) 00002000:00010000:3.0:1788845039.207461:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01dbb100 00000100:00100000:2.0:1788845039.213505:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.213920:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641691264 00000100:00100000:3.0:1788845039.213931:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641691264 00002000:00100000:3.0:1788845039.213981:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1556480->1560575) req@ffff94a03beaad80 x1875739641691264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.213993:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beaad80 with x1875739641691264 ext(1556480->1560575) 00002000:00010000:3.0:1788845039.213999:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beaad80 00000100:00100000:3.0:1788845039.214075:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 451 00000100:00100000:3.0:1788845039.214085:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641691264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845039.214415:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641691264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 330us (919us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845039.214424:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 451 00002000:00100000:3.0:1788845039.214432:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1556480->1560575) req@ffff94a03beaad80 x1875739641691264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/4536 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.214440:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beaad80 with x1875739641691264 ext(1556480->1560575) 00002000:00010000:3.0:1788845039.214443:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beaad80 00000100:00100000:1.0:1788845039.217488:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.217589:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641691392 00000100:00100000:3.0:1788845039.217600:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641691392 00002000:00100000:3.0:1788845039.217716:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1400832->1560575) req@ffff94a035689180 x1875739641691392/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 504/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.217728:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035689180 with x1875739641691392 ext(1400832->1560575) 00002000:00010000:3.0:1788845039.217733:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035689180 00000100:00100000:3.0:1788845039.217817:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 452 00000100:00100000:3.0:1788845039.217828:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641691392:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845039.220856:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641691392:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3018us (3367us total) trans 4294967459 rc 0/0 00000100:00100000:0.0:1788845039.220870:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 452 00002000:00100000:0.0:1788845039.220901:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1400832->1560575) req@ffff94a035689180 x1875739641691392/t4294967459(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 504/448 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.220912:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035689180 with x1875739641691392 ext(1400832->1560575) 00002000:00010000:0.0:1788845039.220917:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035689180 00000100:00100000:3.0:1788845039.224484:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.224587:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641691520 00000100:00100000:2.0:1788845039.224598:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641691520 00000100:00100000:2.0:1788845039.224719:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 938 00000100:00100000:2.0:1788845039.224730:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641691520:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845039.261082:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967459 is committed 00000001:00080000:3.0:1788845039.261105:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967458 is committed 00000100:00100000:1.0:1788845039.261508:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641691520:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 36775us (37033us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845039.261519:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 938 00000100:00100000:2.0:1788845039.266194:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.266367:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641691648 00000100:00100000:3.0:1788845039.266376:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641691648 00000100:00100000:3.0:1788845039.266467:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 939 00000100:00100000:3.0:1788845039.266479:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641691648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.266500:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.266519:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9721 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.266552:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.266561:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 14) 00010000:00010000:3.0:1788845039.266566:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 14) 00010000:00010000:3.0:1788845039.266571:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1531904->1548287] (req 1531904->1548287) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.266609:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 5 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.266639:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 5 pid: 6684 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.266653:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03456d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660338560:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845039.266746:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.269953:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.270556:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641691776 00000100:00100000:1.0:1788845039.270566:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641691776 00010000:00010000:1.0:1788845039.270608:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 6 pid: 6684 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.270665:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 457 00000100:00100000:1.0:1788845039.270675:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641691776:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845039.270690:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845039.270713:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 6 pid: 6684 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.270733:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 6 pid: 6684 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.270769:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 14) 00010000:00010000:1.0:1788845039.270774:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 14) 00010000:00010000:1.0:1788845039.270782:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.270793:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.270832:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.270847:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660338688:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845039.270956:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312c00 going to sleep for 11 seconds 00010000:00010000:0.0:1788845039.271193:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae0b7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1400832->1449983) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a971a expref: 6 pid: 6684 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.271209:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03456d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660338560:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845039.271237:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.271247:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031a00, rc 0) 00000100:00100000:0.0:1788845039.271302:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641691648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4825us (5119us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.271313:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 939 00000100:00100000:3.0:1788845039.275671:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.275945:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660338688:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845039.275969:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845039.276109:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641691904 00000100:00100000:2.0:1788845039.276119:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641691904 00002000:00100000:2.0:1788845039.276169:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1531904->1535999) req@ffff949f03456680 x1875739641691904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.276179:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03456680 with x1875739641691904 ext(1531904->1535999) 00002000:00010000:2.0:1788845039.276184:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03456680 00000100:00100000:2.0:1788845039.276212:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 453 00000100:00100000:2.0:1788845039.276222:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641691904:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845039.276477:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641691776:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5803us (6536us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845039.276486:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 457 00000100:00100000:2.0:1788845039.276835:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641691904:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 614us (1173us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845039.276842:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 453 00002000:00100000:2.0:1788845039.276849:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1531904->1535999) req@ffff949f03456680 x1875739641691904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/4536 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.276865:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03456680 with x1875739641691904 ext(1531904->1535999) 00002000:00010000:2.0:1788845039.276869:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03456680 00000100:00100000:3.0:1788845039.280969:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.281087:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641692032 00000100:00100000:0.0:1788845039.281096:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641692032 00002000:00100000:0.0:1788845039.281145:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1544192->1548287) req@ffff949f03457b80 x1875739641692032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.281156:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03457b80 with x1875739641692032 ext(1544192->1548287) 00002000:00010000:0.0:1788845039.281161:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03457b80 00000100:00100000:0.0:1788845039.281195:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 454 00000100:00100000:0.0:1788845039.281204:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641692032:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845039.281585:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641692032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 381us (623us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845039.281592:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 454 00002000:00100000:0.0:1788845039.281598:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1544192->1548287) req@ffff949f03457b80 x1875739641692032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/4536 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.281606:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03457b80 with x1875739641692032 ext(1544192->1548287) 00002000:00010000:0.0:1788845039.282107:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03457b80 00000100:00100000:1.0:1788845039.285244:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.285490:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641692160 00000100:00100000:0.0:1788845039.285500:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641692160 00000100:00100000:0.0:1788845039.285558:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 940 00000100:00100000:0.0:1788845039.285568:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641692160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.285587:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.285606:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae0c5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9728 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.285676:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660338816:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.285770:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.288088:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660338816:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845039.288127:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae0c5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9728 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.288152:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae0c5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9728 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.288167:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845039.288285:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641692160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2717us (3049us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.288294:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 940 00000100:00100000:0.0:1788845039.291867:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.292018:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641692288 00000100:00100000:2.0:1788845039.292029:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641692288 00000100:00100000:2.0:1788845039.292117:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 941 00000100:00100000:2.0:1788845039.292128:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641692288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.292147:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.292165:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae0cc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a972f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.292226:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660338944:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.292422:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.294564:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660338944:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.294605:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae0cc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a972f expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.294642:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae0cc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a972f expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.294659:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845039.294730:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641692288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2603us (2872us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.294748:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 941 00000100:00100000:3.0:1788845039.297706:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.297862:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641692544 00000100:00100000:1.0:1788845039.297871:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641692544 00000100:00100000:1.0:1788845039.297971:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1319 00000100:00100000:1.0:1788845039.297981:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:14891:x1875739641692544:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845039.298039:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845039.298068:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae094 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96f7 expref: 29 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.298089:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.298093:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845039.298322:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.298338:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.298348:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.298383:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.298402:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 11 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.298417:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660339072:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845039.298495:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.301705:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.301790:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641692672 00000100:00100000:2.0:1788845039.301800:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641692672 00010000:00010000:2.0:1788845039.301843:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 12 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.301923:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 458 00000100:00100000:2.0:1788845039.301933:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641692672:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845039.301952:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845039.301959:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 12 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.301980:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 12 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.302002:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.302007:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.302018:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.302027:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.302047:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845039.302095:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641692672:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 164us (400us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.302103:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 458 00010000:00010000:1.0:1788845039.302254:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae055 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96bf expref: 11 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.302282:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660339072:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845039.302302:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.302312:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.302318:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845039.302431:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967769 00000004:00010000:1.0:1788845039.302482:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 724182 -> 602308 00000004:00010000:1.0:1788845039.302486:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.302488:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845038 -> 1788845036 00000004:00010000:1.0:1788845039.302490:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.302492:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:1.0:1788845039.302499:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d9e7c9ed reply state 00000000635838a4 transno 4294967770 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.302506:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845039.302574:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+31:14891:x1875739641692544:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4594us (4877us total) trans 4294967770 rc 0/0 00000100:00100000:1.0:1788845039.302584:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1319 00000020:00080000:1.0:1788845039.302588:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845039.304525:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.304542:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.304548:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.304555:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845039.304559:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.304567:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.304583:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.304586:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845039.304590:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.304595:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae0d3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.306225:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.306359:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641692800 00000100:00100000:1.0:1788845039.306369:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641692800 00000100:00100000:1.0:1788845039.306425:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 942 00000100:00100000:1.0:1788845039.306436:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641692800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.306455:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.306474:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9736 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.306512:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.306523:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 14) 00010000:00010000:1.0:1788845039.306527:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 14) 00010000:00010000:1.0:1788845039.306533:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.306570:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.306589:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 5 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.306604:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660339200:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845039.306703:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.310299:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660339200:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845039.310336:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.310358:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:1.0:1788845039.310419:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641692800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3985us (4202us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845039.310428:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 942 00000100:00100000:3.0:1788845039.312334:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.312455:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641692928 00000100:00100000:2.0:1788845039.312465:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641692928 00002000:00100000:2.0:1788845039.312512:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1531904->1548287) req@ffff949f03456d80 x1875739641692928/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 16872/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.312522:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03456d80 with x1875739641692928 ext(1531904->1548287) 00010000:00010000:2.0:1788845039.312531:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 6 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.312552:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 6 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845039.312559:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f03456d80 00000100:00100000:2.0:1788845039.312594:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 26 00000100:00100000:2.0:1788845039.312604:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641692928:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845039.313119:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641692928:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 515us (791us total) trans 4294967460 rc 0/0 00000100:00100000:2.0:1788845039.313127:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 26 00002000:00100000:2.0:1788845039.313133:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1531904->1548287) req@ffff949f03456d80 x1875739641692928/t4294967460(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 16872/448 e 0 to 0 dl 1788845050 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.313142:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03456d80 with x1875739641692928 ext(1531904->1548287) 00010000:00010000:2.0:1788845039.313148:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 8 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.313161:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 8 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845039.313168:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f03456d80 00000100:00100000:3.0:1788845039.317142:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.317247:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641693056 00000100:00100000:2.0:1788845039.317265:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641693056 00010000:00010000:2.0:1788845039.317307:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 8 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.317352:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 459 00000100:00100000:2.0:1788845039.317362:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641693056:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845039.317381:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845039.317401:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 8 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.317419:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 8 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.317443:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae0be lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1548287) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9721 expref: 8 pid: 6682 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.317461:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 14) 00010000:00010000:2.0:1788845039.317465:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 14) 00010000:00010000:2.0:1788845039.317472:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.317481:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.317515:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.317529:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660339328:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845039.317762:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.321105:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660339328:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845039.321131:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845039.321192:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641693056:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3830us (4058us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.321203:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 459 00000100:00100000:2.0:1788845039.323717:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.323812:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641693184 00000100:00100000:3.0:1788845039.323822:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641693184 00002000:00010000:3.0:1788845039.323867:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1914566->18446744073709551615) 00002000:00100000:3.0:1788845039.323871:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f826a00 with x1875739641693184 ext(1914566->18446744073709551615) 00002000:00010000:3.0:1788845039.323921:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f826a00. 00000100:00100000:3.0:1788845039.323946:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 455 00000100:00100000:3.0:1788845039.323956:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641693184:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845039.324210:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1875067 -> 1914566 00002000:00010000:3.0:1788845039.324214:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00002000:00010000:3.0:1788845039.324217:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845038 -> 1788845038 00002000:00010000:3.0:1788845039.324220:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00002000:00010000:3.0:1788845039.324222:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 317 -> 145 00000100:00100000:3.0:1788845039.324311:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641693184:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 356us (604us total) trans 4294967461 rc 0/0 00000100:00100000:3.0:1788845039.324320:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 455 00002000:00010000:3.0:1788845039.324324:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1914566->18446744073709551615) 00002000:00100000:3.0:1788845039.324327:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f826a00 with x1875739641693184 ext(1914566->18446744073709551615) 00002000:00010000:3.0:1788845039.324332:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f826a00. 00000100:00100000:3.0:1788845039.328209:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.328577:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641693312 00000100:00100000:2.0:1788845039.328588:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641693312 00002000:00100000:2.0:1788845039.328656:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1859584->1863679) req@ffff94a035743480 x1875739641693312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.328669:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035743480 with x1875739641693312 ext(1859584->1863679) 00002000:00010000:2.0:1788845039.328674:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035743480 00000100:00100000:2.0:1788845039.328720:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 456 00000100:00100000:2.0:1788845039.328730:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641693312:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845039.329047:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641693312:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 316us (845us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845039.329054:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 456 00002000:00100000:2.0:1788845039.329061:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1859584->1863679) req@ffff94a035743480 x1875739641693312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/4536 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.329069:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035743480 with x1875739641693312 ext(1859584->1863679) 00002000:00010000:2.0:1788845039.329073:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035743480 00000100:00100000:1.0:1788845039.335021:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.335118:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641693440 00000100:00100000:3.0:1788845039.335129:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641693440 00002000:00100000:3.0:1788845039.335182:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1474560->1916927) req@ffff94a03568a300 x1875739641693440/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 504/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.335194:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03568a300 with x1875739641693440 ext(1474560->1916927) 00002000:00010000:3.0:1788845039.335200:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03568a300 00000100:00100000:3.0:1788845039.335235:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 457 00000100:00100000:3.0:1788845039.335246:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641693440:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845039.346751:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641693440:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 11486us (11719us total) trans 0 rc 438272/438272 00000100:00100000:2.0:1788845039.346767:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 457 00002000:00100000:2.0:1788845039.346781:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1474560->1916927) req@ffff94a03568a300 x1875739641693440/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 504/440 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 438272/438272 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.346793:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03568a300 with x1875739641693440 ext(1474560->1916927) 00002000:00010000:2.0:1788845039.346800:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03568a300 00000100:00100000:1.0:1788845039.353373:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.353538:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641693568 00000100:00100000:2.0:1788845039.353549:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641693568 00002000:00100000:2.0:1788845039.353603:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1859584->1914565) req@ffff94a035688a80 x1875739641693568/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.353681:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035688a80 with x1875739641693568 ext(1859584->1914565) 00002000:00010000:2.0:1788845039.353688:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035688a80 00000100:00100000:2.0:1788845039.353722:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 458 00000100:00100000:2.0:1788845039.353742:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641693568:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845039.359175:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641693568:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5428us (5797us total) trans 4294967462 rc 0/0 00000100:00100000:2.0:1788845039.359192:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 458 00002000:00100000:2.0:1788845039.359211:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1859584->1914565) req@ffff94a035688a80 x1875739641693568/t4294967462(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/448 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.359223:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035688a80 with x1875739641693568 ext(1859584->1914565) 00002000:00010000:2.0:1788845039.359230:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035688a80 00000100:00100000:3.0:1788845039.361867:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.362044:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641693696 00000100:00100000:2.0:1788845039.362056:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641693696 00000100:00100000:2.0:1788845039.362124:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 943 00000100:00100000:2.0:1788845039.362136:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641693696:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845039.396696:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967462 is committed 00000001:00080000:1.0:1788845039.396717:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967460 is committed 00000100:00100000:2.0:1788845039.397021:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641693696:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 34882us (35161us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.397035:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 943 00000100:00100000:3.0:1788845039.399768:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.399910:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641693824 00000100:00100000:2.0:1788845039.399921:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641693824 00000100:00100000:2.0:1788845039.400041:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1320 00000100:00100000:2.0:1788845039.400053:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+30:14891:x1875739641693824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.400078:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.400100:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae0e1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a973d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.400196:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.400212:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.400225:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845039.400262:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1914566 00000004:00200000:2.0:1788845039.400267:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845039.400288:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.400317:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae0e1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 32 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.400340:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 31 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.400349:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.400353:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845039.400355:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845039.400434:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+31:14891:x1875739641693824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 384us (676us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.400441:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1320 00000100:00100000:0.0:1788845039.412682:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.412728:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.412879:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641693952 00000100:00100000:1.0:1788845039.412921:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641693952 00002000:00100000:1.0:1788845039.412973:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c1500 x1875739641693952/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 504/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.412984:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c1500 with x1875739641693952 00000100:00100000:1.0:1788845039.413022:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 878 00000100:00100000:1.0:1788845039.413031:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641693952:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845039.416951:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641694080 00000100:00100000:2.0:1788845039.416969:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641694080 00002000:00100000:2.0:1788845039.417025:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1474559) req@ffff94a03f6c3100 x1875739641694080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.417038:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c3100 with x1875739641694080 ext(1048576->1474559) 00002000:00010000:2.0:1788845039.417043:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c3100 00000100:00100000:2.0:1788845039.417074:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 459 00000100:00100000:2.0:1788845039.417083:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641694080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845039.428064:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641693952:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 15022us (15381us total) trans 0 rc 606208/606208 00000100:00100000:1.0:1788845039.428079:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 878 00002000:00100000:1.0:1788845039.428094:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c1500 x1875739641693952/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 504/440 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 606208/606208 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.428113:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c1500 with x1875739641693952 00000100:00100000:2.0:1788845039.431801:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641694080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14707us (19060us total) trans 0 rc 425984/425984 00000100:00100000:2.0:1788845039.431815:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 459 00002000:00100000:2.0:1788845039.431828:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1474559) req@ffff94a03f6c3100 x1875739641694080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/440 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 425984/425984 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.431839:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c3100 with x1875739641694080 ext(1048576->1474559) 00002000:00010000:2.0:1788845039.431845:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c3100 00000100:00100000:0.0:1788845039.436232:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.436527:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641694208 00000100:00100000:2.0:1788845039.436538:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641694208 00000100:00100000:2.0:1788845039.436698:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1321 00000100:00100000:2.0:1788845039.436709:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641694208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.436739:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.436763:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae0ef lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9744 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.436860:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae0f6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.436875:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae0f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.436989:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae0f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845039.437019:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1914566 00000004:00200000:2.0:1788845039.437023:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845039.437043:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae0f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.437069:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae0ef lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9744 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.437087:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae0f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9744 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.437096:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.437099:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845039.437102:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845039.437206:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641694208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 498us (985us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.437213:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1321 00000100:00100000:0.0:1788845039.440235:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.440367:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.440456:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641694336 00000100:00100000:1.0:1788845039.440465:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641694336 00000100:00100000:1.0:1788845039.440556:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1322 00000100:00100000:1.0:1788845039.440568:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641694336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.440588:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.440618:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae0fd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a974b expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.440653:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.440693:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660339456:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.440786:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.442548:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641694464 00000100:00100000:3.0:1788845039.442559:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641694464 00000004:00010000:0.0:1788845039.442812:0:11036:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 602308 -> 724182 00000004:00010000:0.0:1788845039.442817:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845039 00000100:00100000:0.0:1788845039.442841:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660339456:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.442866:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:0.0:1788845039.442912:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae0fd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a974b expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.442933:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.442936:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.442941:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae0fd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a974b expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.442961:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:0.0:1788845039.443020:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641694336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2454us (2795us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.443027:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1322 00000100:00100000:1.0:1788845039.443137:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 944 00000100:00100000:1.0:1788845039.443151:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641694464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.443169:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.443187:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae104 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9752 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.443246:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660339584:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.446916:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00002000:00010000:1.0:1788845039.447584:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845038 -> 1788845039 00002000:00010000:1.0:1788845039.447589:0:7983:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 145 -> 293 00000100:00100000:1.0:1788845039.447674:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660339584:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845039.447709:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae104 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9752 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.447744:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae104 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9752 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.447758:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845039.447853:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641694464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4705us (7486us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.447863:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 944 00000100:00100000:0.0:1788845039.451037:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.451085:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.451161:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641694592 00000100:00100000:3.0:1788845039.451172:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641694592 00000100:00100000:3.0:1788845039.451232:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1323 00000100:00100000:3.0:1788845039.451242:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641694592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.451275:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.451299:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae10b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9759 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.451336:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.451377:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660339712:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.451455:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.451814:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641694720 00000100:00100000:1.0:1788845039.451819:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641694720 00000100:00100000:1.0:1788845039.451862:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 945 00000100:00100000:1.0:1788845039.451870:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641694720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.451880:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.451916:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae112 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9760 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.451955:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660339840:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.451986:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.455420:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660339712:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845039.455450:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:3.0:1788845039.455469:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae10b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9759 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.455490:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.455493:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845039.455498:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae10b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9759 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.455513:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:3.0:1788845039.455574:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641694592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4332us (4487us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.455582:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1323 00000100:00100000:1.0:1788845039.457530:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660339840:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845039.457566:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae112 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9760 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.457591:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae112 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9760 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.457605:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845039.457688:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641694720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5817us (6660us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.457697:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 945 00000100:00100000:3.0:1788845039.462325:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.462372:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.462506:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641694976 00000100:00100000:0.0:1788845039.462516:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641694976 00000100:00100000:0.0:1788845039.462570:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 946 00000100:00100000:0.0:1788845039.462578:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641694976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.462596:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.462621:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae119 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a976e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.462677:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660339968:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.462769:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.462920:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641694848 00000100:00100000:2.0:1788845039.462925:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641694848 00000100:00100000:2.0:1788845039.462960:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1324 00000100:00100000:2.0:1788845039.462967:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641694848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.462977:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.462989:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae120 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9767 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.463015:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.463039:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f827100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660340096:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.463067:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.466142:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f827100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660340096:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845039.466172:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:2.0:1788845039.466190:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae120 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9767 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.466212:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.466215:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845039.466221:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae120 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9767 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.466243:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:2.0:1788845039.466312:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641694848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3344us (3997us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.466321:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1324 00000100:00100000:0.0:1788845039.466432:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660339968:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845039.466454:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae119 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a976e expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.466470:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae119 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a976e expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.466481:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845039.466505:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641694976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3929us (4134us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.466512:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 946 00000100:00100000:3.0:1788845039.470372:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.470509:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641695104 00000100:00100000:0.0:1788845039.470519:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641695104 00000100:00100000:0.0:1788845039.470583:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1325 00000100:00100000:0.0:1788845039.470593:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641695104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.470623:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.470648:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae127 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9775 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.470690:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.470744:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660340224:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.470853:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.473369:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.473577:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641695232 00000100:00100000:0.0:1788845039.473587:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641695232 00000100:00100000:0.0:1788845039.473661:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 947 00000100:00100000:0.0:1788845039.473671:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641695232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.473692:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.473710:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae12e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a977c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.473782:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660340352:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.473865:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.476312:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660340352:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.476361:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae12e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a977c expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.476386:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae12e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a977c expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.476399:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:2.0:1788845039.476457:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641695232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2787us (3097us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.476465:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 947 00000100:00100000:0.0:1788845039.477087:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660340224:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.477103:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:0.0:1788845039.477115:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae127 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9775 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.477129:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.477131:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.477135:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae127 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9775 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.477146:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845039.477190:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641695104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6598us (6828us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.477196:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1325 00000100:00100000:1.0:1788845039.482448:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.482525:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641695488 00000100:00100000:0.0:1788845039.482535:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641695488 00000100:00100000:0.0:1788845039.482595:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1326 00000100:00100000:0.0:1788845039.482604:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641695488:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845039.482679:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845039.482706:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae0f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9744 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.482723:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.482728:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845039.482980:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.482995:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 30 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.483004:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.483037:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 30 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.483054:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 30 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.483067:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660340480:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845039.483153:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.486591:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.486744:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641695616 00000100:00100000:2.0:1788845039.486754:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641695616 00010000:00010000:2.0:1788845039.486793:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 31 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.486841:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 460 00000100:00100000:2.0:1788845039.486850:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d11850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641695616:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845039.486864:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845039.486872:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 31 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.486923:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 31 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.486945:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.486950:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.486960:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.486967:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.486984:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845039.487031:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d11850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641695616:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 182us (449us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.487038:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 460 00010000:00010000:0.0:1788845039.487232:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae0e8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a973d expref: 30 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.487250:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660340480:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845039.487269:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.487277:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.487283:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845039.487393:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967771 00000004:00010000:0.0:1788845039.487443:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 724182 -> 602308 00000004:00010000:0.0:1788845039.487447:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.487450:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845036 00000004:00010000:0.0:1788845039.487453:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.487455:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:0.0:1788845039.487462:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004b48439e reply state 000000001fd3afbd transno 4294967772 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.487471:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845039.487543:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641695488:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4940us (5104us total) trans 4294967772 rc 0/0 00000100:00100000:0.0:1788845039.487550:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1326 00000020:00080000:0.0:1788845039.487554:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845039.490932:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.490952:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.490960:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.490966:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845039.490971:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.490979:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.490996:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.491000:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845039.491004:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.491016:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287ae135 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.491671:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.492042:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641695744 00000100:00100000:1.0:1788845039.492052:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641695744 00000100:00100000:1.0:1788845039.492113:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1327 00000100:00100000:1.0:1788845039.492124:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641695744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.492149:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.492170:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9783 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.492198:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.492209:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.492242:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.492273:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 29 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.492293:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660340608:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845039.492374:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.495526:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.495637:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641695872 00000100:00100000:2.0:1788845039.495648:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641695872 00002000:00100000:2.0:1788845039.495699:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d6680 x1875739641695872/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.495710:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d6680 with x1875739641695872 00010000:00010000:2.0:1788845039.495718:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 30 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.495746:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 30 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845039.495756:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a0359d6680 00000100:00100000:2.0:1788845039.495792:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 32 00000100:00100000:2.0:1788845039.495802:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+30:2370:x1875739641695872:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845039.496656:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660340608:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845039.496690:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.496705:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.496708:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.496713:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.496723:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845039.496814:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641695744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4692us (5148us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845039.496822:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1327 00000004:00010000:1.0:1788845039.499204:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 602308 -> 724182 00000004:00010000:1.0:1788845039.499213:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.499216:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845039.499218:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.499220:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845039.499290:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641695872:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3485us (3770us total) trans 4294967773 rc 0/0 00000100:00100000:1.0:1788845039.499300:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 32 00002000:00100000:1.0:1788845039.499312:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d6680 x1875739641695872/t4294967773(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/448 e 0 to 0 dl 1788845050 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.499323:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d6680 with x1875739641695872 00010000:00010000:1.0:1788845039.499333:0:5854:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 32 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.499354:0:5854:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 32 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845039.499362:0:5854:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a0359d6680 00000100:00100000:3.0:1788845039.505347:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.505456:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641696000 00000100:00100000:1.0:1788845039.505467:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641696000 00010000:00010000:1.0:1788845039.505509:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 32 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.505553:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 461 00000100:00100000:1.0:1788845039.505563:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:2367:x1875739641696000:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845039.505579:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845039.505607:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 32 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.505636:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 32 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.505659:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae047 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a96b8 expref: 32 pid: 11036 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.505676:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.505683:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.505694:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.505703:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.505761:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.505775:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660340736:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845039.505864:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.509168:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.509293:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660340736:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845039.509323:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845039.509389:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+31:2367:x1875739641696000:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3827us (4054us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845039.509401:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 461 00000100:00100000:3.0:1788845039.509939:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641696128 00000100:00100000:3.0:1788845039.509948:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641696128 00000100:00100000:3.0:1788845039.509992:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 948 00000100:00100000:3.0:1788845039.510002:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641696128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.510020:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.510038:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a978a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.510076:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.510086:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 14) 00010000:00010000:3.0:1788845039.510090:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 14) 00010000:00010000:3.0:1788845039.510096:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.510129:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.510147:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 5 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.510160:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660340864:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845039.510227:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.515045:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.515314:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641696256 00000100:00100000:0.0:1788845039.515325:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641696256 00010000:00010000:0.0:1788845039.515367:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 6 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.515412:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 462 00000100:00100000:0.0:1788845039.515424:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641696256:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845039.515440:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845039.515463:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 6 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.515482:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 6 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.515508:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 14) 00010000:00010000:0.0:1788845039.515513:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 14) 00010000:00010000:0.0:1788845039.515522:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.515533:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.515575:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.515591:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660340992:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845039.515685:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788845039.516267:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005931e884/0xb2187301287ae0da lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9736 expref: 6 pid: 7983 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.516284:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660340864:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845039.516309:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.516329:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:3.0:1788845039.516378:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641696128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6377us (7220us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845039.516386:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 948 00000100:00100000:2.0:1788845039.521447:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.521585:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660340992:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845039.521620:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845039.521681:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641696256:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6258us (6647us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.521692:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 462 00000100:00100000:3.0:1788845039.522063:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641696384 00000100:00100000:3.0:1788845039.522071:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641696384 00002000:00010000:3.0:1788845039.522110:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1887150->18446744073709551615) 00002000:00100000:3.0:1788845039.522115:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03456300 with x1875739641696384 ext(1887150->18446744073709551615) 00002000:00010000:3.0:1788845039.522125:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03456300. 00000100:00100000:3.0:1788845039.522156:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 460 00000100:00100000:3.0:1788845039.522166:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641696384:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845039.522581:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1914566 -> 1887150 00002000:00010000:3.0:1788845039.522586:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00002000:00010000:3.0:1788845039.522588:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845039 -> 1788845038 00002000:00010000:3.0:1788845039.522590:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00002000:00010000:3.0:1788845039.522592:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:3.0:1788845039.522742:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641696384:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 566us (1295us total) trans 4294967463 rc 0/0 00000100:00100000:3.0:1788845039.522750:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 460 00002000:00010000:3.0:1788845039.522754:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1887150->18446744073709551615) 00002000:00100000:3.0:1788845039.522757:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03456300 with x1875739641696384 ext(1887150->18446744073709551615) 00002000:00010000:3.0:1788845039.522761:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03456300. 00000100:00100000:2.0:1788845039.527024:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.527112:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641696512 00000100:00100000:3.0:1788845039.527123:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641696512 00000100:00100000:3.0:1788845039.527184:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1328 00000100:00100000:3.0:1788845039.527195:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641696512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.527216:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.527238:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae14a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9791 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.527341:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae151 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.527357:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae151 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.527825:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae151 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845039.527859:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1887150 00000004:00200000:3.0:1788845039.527863:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845039.527920:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae151 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.527950:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae14a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9791 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.527968:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae151 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9791 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.527977:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.527980:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845039.527983:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:3.0:1788845039.528140:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641696512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 947us (1126us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845039.528148:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1328 00000100:00100000:1.0:1788845039.538308:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.538420:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641696640 00000100:00100000:0.0:1788845039.538430:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641696640 00002000:00100000:0.0:1788845039.538481:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1888255) req@ffff94a035689880 x1875739641696640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.538493:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035689880 with x1875739641696640 ext(1048576->1888255) 00002000:00010000:0.0:1788845039.538499:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035689880 00000100:00100000:0.0:1788845039.538537:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 461 00000100:00100000:0.0:1788845039.538547:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641696640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845039.545427:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.545538:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641696768 00000100:00100000:3.0:1788845039.545549:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641696768 00002000:00100000:3.0:1788845039.545598:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d2680 x1875739641696768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.545619:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d2680 with x1875739641696768 00000100:00100000:3.0:1788845039.545659:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 879 00000100:00100000:3.0:1788845039.545669:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641696768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845039.556303:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641696640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17745us (17994us total) trans 0 rc 839680/839680 00000100:00100000:0.0:1788845039.556319:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 461 00002000:00100000:0.0:1788845039.556334:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1888255) req@ffff94a035689880 x1875739641696640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/440 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 839680/839680 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.556350:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035689880 with x1875739641696640 ext(1048576->1888255) 00002000:00010000:0.0:1788845039.556357:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035689880 00000100:00100000:3.0:1788845039.561860:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641696768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 16181us (16431us total) trans 0 rc 118784/118784 00000100:00100000:3.0:1788845039.561874:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 879 00002000:00100000:3.0:1788845039.561932:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d2680 x1875739641696768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/440 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 118784/118784 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.561943:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d2680 with x1875739641696768 00000100:00100000:0.0:1788845039.563544:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.563709:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641696896 00000100:00100000:2.0:1788845039.563719:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641696896 00000100:00100000:2.0:1788845039.563786:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1329 00000100:00100000:2.0:1788845039.563797:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:14891:x1875739641696896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.563824:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.563845:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae158 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9798 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.563976:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.563992:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.564004:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845039.564032:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1887150 00000004:00200000:2.0:1788845039.564037:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845039.564057:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.564085:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032000/0xb2187301287ae158 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 33 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.564104:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 32 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.564113:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.564116:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845039.564120:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033800, rc 0) 00000100:00100000:2.0:1788845039.564184:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+32:14891:x1875739641696896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 389us (647us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.564192:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1329 00000100:00100000:2.0:1788845039.569156:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.569198:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.569292:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641697024 00000100:00100000:0.0:1788845039.569303:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641697024 00000100:00100000:0.0:1788845039.569374:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1330 00000100:00100000:0.0:1788845039.569385:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641697024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.569407:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.569430:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae166 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a979f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.569468:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.569511:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660341120:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.569602:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.570051:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641697152 00000100:00100000:1.0:1788845039.570057:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641697152 00000100:00100000:1.0:1788845039.570092:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 949 00000100:00100000:1.0:1788845039.570101:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641697152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.570113:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.570126:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ae16d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97a6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.570172:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660341248:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.570215:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00002000:00010000:3.0:1788845039.575075:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845038 -> 1788845039 00000100:00100000:3.0:1788845039.575107:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660341248:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845039.575146:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ae16d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97a6 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.575172:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b15a3072/0xb2187301287ae16d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97a6 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.575191:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:3.0:1788845039.575260:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641697152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5152us (6052us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.575269:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 949 00000004:00010000:0.0:1788845039.576124:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845039 00000100:00100000:0.0:1788845039.576151:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660341120:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.576168:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:0.0:1788845039.576184:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae166 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a979f expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.576200:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.576203:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.576208:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae166 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a979f expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.576221:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845039.576284:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641697024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6900us (7141us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.576292:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1330 00000100:00100000:2.0:1788845039.580011:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.580055:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.580157:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641697408 00000100:00100000:3.0:1788845039.580167:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641697408 00000100:00100000:3.0:1788845039.580228:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 950 00000100:00100000:3.0:1788845039.580239:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641697408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.580269:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.580287:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae174 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97b4 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.580347:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660341376:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.580426:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.581000:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641697280 00000100:00100000:0.0:1788845039.581005:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641697280 00000100:00100000:0.0:1788845039.581050:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1331 00000100:00100000:0.0:1788845039.581059:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641697280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.581072:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.581087:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae17b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97ad expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.581118:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.581146:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660341504:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.581189:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.584674:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660341376:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845039.584717:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae174 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97b4 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.584754:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae174 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97b4 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.584769:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:3.0:1788845039.584829:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641697408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4591us (4771us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.584839:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 950 00000100:00100000:0.0:1788845039.585013:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660341504:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.585028:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:0.0:1788845039.585040:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae17b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97ad expref: 33 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.585053:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.585056:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.585060:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae17b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97ad expref: 33 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.585072:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845039.585096:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641697280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4039us (5097us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.585104:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1331 00000100:00100000:2.0:1788845039.589023:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.589073:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.589182:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641697536 00000100:00100000:0.0:1788845039.589192:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641697536 00000100:00100000:0.0:1788845039.589245:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1332 00000100:00100000:0.0:1788845039.589266:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641697536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.589288:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.589308:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287ae182 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97bb expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.589346:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.589386:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660341632:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.589471:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.590175:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641697664 00000100:00100000:3.0:1788845039.590182:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641697664 00000100:00100000:3.0:1788845039.590218:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 951 00000100:00100000:3.0:1788845039.590226:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641697664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.590238:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.590250:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287ae189 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97c2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.590301:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660341760:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.590355:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.593859:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660341760:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845039.593948:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287ae189 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97c2 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.593972:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287ae189 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97c2 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.593986:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030600, rc 0) 00000100:00100000:3.0:1788845039.594044:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641697664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3818us (4970us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.594052:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 951 00000100:00100000:0.0:1788845039.594160:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660341632:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.594175:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:0.0:1788845039.594188:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287ae182 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97bb expref: 33 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.594202:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.594205:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.594209:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287ae182 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97bb expref: 33 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.594221:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007800, rc 0) 00000100:00100000:0.0:1788845039.594246:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641697536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4993us (5235us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.594267:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1332 00000100:00100000:2.0:1788845039.597969:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.598065:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641697792 00000100:00100000:3.0:1788845039.598077:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641697792 00000100:00100000:3.0:1788845039.598137:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1333 00000100:00100000:3.0:1788845039.598149:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+32:2367:x1875739641697792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.598172:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.598193:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae190 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97c9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.598238:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.598294:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660341888:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.598376:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.600473:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.600555:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641697920 00000100:00100000:3.0:1788845039.600565:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641697920 00000100:00100000:3.0:1788845039.600628:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 952 00000100:00100000:3.0:1788845039.600638:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641697920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.600655:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.600672:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae197 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97d0 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.600739:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660342016:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.600815:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.603750:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660342016:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845039.603790:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae197 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97d0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.603815:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae197 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97d0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.603828:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031a00, rc 0) 00000100:00100000:0.0:1788845039.603929:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641697920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3292us (3462us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.603937:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 952 00000100:00100000:3.0:1788845039.604108:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660341888:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845039.604121:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:3.0:1788845039.604133:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae190 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97c9 expref: 33 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.604147:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.604150:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845039.604155:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae190 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97c9 expref: 33 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.604166:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:3.0:1788845039.604193:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+32:2367:x1875739641697792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6047us (6236us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.604201:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1333 00000100:00100000:2.0:1788845039.609128:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.609221:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641698176 00000100:00100000:1.0:1788845039.609233:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641698176 00000100:00100000:1.0:1788845039.609311:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1334 00000100:00100000:1.0:1788845039.609322:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641698176:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845039.609379:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845039.609409:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae151 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9791 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.609426:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.609430:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845039.609653:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.609670:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 31 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.609677:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.609710:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 31 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.609728:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 31 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.609759:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660342144:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845039.609841:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.614560:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.614715:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660342144:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845039.614752:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.614818:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641698304 00000100:00100000:3.0:1788845039.614825:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641698304 00010000:00010000:3.0:1788845039.614863:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 32 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.614934:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 463 00000100:00100000:3.0:1788845039.614944:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641698304:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845039.614959:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845039.614966:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 32 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.614982:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 32 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.615003:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033800/0xb2187301287ae15f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9798 expref: 32 pid: 5840 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.615019:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.615023:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.615032:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.615039:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.615059:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845039.615111:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641698304:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 168us (559us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845039.615118:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 463 00010000:00010000:0.0:1788845039.615187:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.615194:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845039.615322:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967774 00000004:00010000:0.0:1788845039.615373:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 724182 -> 724182 00000004:00010000:0.0:1788845039.615378:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.615380:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845036 00000004:00010000:0.0:1788845039.615383:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.615384:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:0.0:1788845039.615391:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000082ade3b4 reply state 000000006e7a36dc transno 4294967775 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.615399:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845039.615465:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641698176:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6145us (6347us total) trans 4294967775 rc 0/0 00000100:00100000:0.0:1788845039.615472:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1334 00000020:00080000:0.0:1788845039.615475:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845039.618132:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.618146:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.618151:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.618154:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d15400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845039.618158:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.618163:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.618176:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.618178:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845039.618181:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.618185:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae19e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.619006:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.619057:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641698432 00000100:00100000:0.0:1788845039.619067:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641698432 00002000:00010000:0.0:1788845039.619108:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1459157->18446744073709551615) 00002000:00100000:0.0:1788845039.619111:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beaa300 with x1875739641698432 ext(1459157->18446744073709551615) 00002000:00010000:0.0:1788845039.619122:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beaa300. 00000100:00100000:0.0:1788845039.619149:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 462 00000100:00100000:0.0:1788845039.619156:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641698432:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845039.619567:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1887150 -> 1459157 00002000:00010000:0.0:1788845039.619571:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00002000:00010000:0.0:1788845039.619573:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845039 -> 1788845038 00002000:00010000:0.0:1788845039.619574:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00002000:00010000:0.0:1788845039.619576:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:0.0:1788845039.619637:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641698432:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 480us (636us total) trans 4294967464 rc 0/0 00000100:00100000:0.0:1788845039.619642:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 462 00002000:00010000:0.0:1788845039.619645:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1459157->18446744073709551615) 00002000:00100000:0.0:1788845039.619647:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beaa300 with x1875739641698432 ext(1459157->18446744073709551615) 00002000:00010000:0.0:1788845039.619651:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beaa300. 00000100:00100000:2.0:1788845039.624771:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.624855:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641698560 00000100:00100000:0.0:1788845039.624866:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641698560 00000100:00100000:0.0:1788845039.624959:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1335 00000100:00100000:0.0:1788845039.624970:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641698560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.624995:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.625017:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287ae1a5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97d7 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.625103:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae1ac lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.625118:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae1ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.625130:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae1ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845039.625154:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1459157 00000004:00200000:0.0:1788845039.625157:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845039.625174:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae1ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.625197:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287ae1a5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97d7 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.625214:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae1ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97d7 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.625222:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.625224:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845039.625227:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845039.625294:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641698560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 326us (532us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.625301:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1335 00000100:00100000:2.0:1788845039.628296:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.628375:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641698688 00000100:00100000:0.0:1788845039.628385:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641698688 00000100:00100000:0.0:1788845039.628449:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1336 00000100:00100000:0.0:1788845039.628460:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+31:14891:x1875739641698688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.628483:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.628501:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae1b3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97de expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.628586:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.628600:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.628621:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845039.628646:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1459157 00000004:00200000:0.0:1788845039.628650:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845039.628665:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.628689:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae1b3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 33 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.628707:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 32 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.628716:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.628719:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845039.628722:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845039.628793:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:14891:x1875739641698688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 334us (506us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.628800:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1336 00000100:00100000:2.0:1788845039.631518:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.631599:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641698816 00000100:00100000:0.0:1788845039.631620:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641698816 00000100:00100000:0.0:1788845039.631677:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1337 00000100:00100000:0.0:1788845039.631686:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641698816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.631709:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.631741:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1c1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97e5 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.631781:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.631824:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660342272:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.631930:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000004:00010000:0.0:1788845039.634049:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845039 00000100:00100000:0.0:1788845039.634087:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660342272:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.634110:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:0.0:1788845039.634130:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1c1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97e5 expref: 33 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.634149:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.634152:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.634156:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97e5 expref: 33 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.634170:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:0.0:1788845039.634229:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641698816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2543us (2720us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.634240:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1337 00000100:00100000:2.0:1788845039.636078:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.636163:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641698944 00000100:00100000:1.0:1788845039.636172:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641698944 00000100:00100000:1.0:1788845039.636231:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 953 00000100:00100000:1.0:1788845039.636241:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641698944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.636270:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.636290:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae1c8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97ec expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.636349:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660342400:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.636431:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845039.643008:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845038 -> 1788845039 00000100:00100000:1.0:1788845039.643041:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660342400:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845039.643077:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae1c8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97ec expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.643103:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae1c8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97ec expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.643121:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845039.643198:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641698944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6958us (7127us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.643208:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 953 00000100:00100000:2.0:1788845039.647231:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.647289:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.647414:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641699200 00000100:00100000:3.0:1788845039.647423:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641699200 00000100:00100000:3.0:1788845039.647482:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 954 00000100:00100000:3.0:1788845039.647492:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641699200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.647514:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.647534:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae1cf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97fa expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.647596:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660342528:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.647690:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.648467:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641699072 00000100:00100000:0.0:1788845039.648474:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641699072 00000100:00100000:0.0:1788845039.648521:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1338 00000100:00100000:0.0:1788845039.648530:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641699072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.648544:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.648560:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1d6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a97f3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.648592:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.648632:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660342656:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.648690:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.654874:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660342528:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845039.654962:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae1cf lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97fa expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.654987:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae1cf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97fa expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.655007:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:3.0:1788845039.655063:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641699200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7573us (7773us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.655071:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 954 00000100:00100000:0.0:1788845039.655187:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02383800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660342656:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.655202:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:0.0:1788845039.655214:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1d6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97f3 expref: 33 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.655227:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.655230:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.655234:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1d6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97f3 expref: 33 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.655245:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:0.0:1788845039.655279:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641699072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6750us (8059us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.655286:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1338 00000100:00100000:3.0:1788845039.659716:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.659772:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.659938:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641699328 00000100:00100000:1.0:1788845039.659949:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641699328 00000100:00100000:1.0:1788845039.660006:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1339 00000100:00100000:1.0:1788845039.660016:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641699328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.660040:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.660061:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae1dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9801 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.660099:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.660141:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660342784:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.660225:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.661209:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641699456 00000100:00100000:2.0:1788845039.661219:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641699456 00000100:00100000:2.0:1788845039.661282:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 955 00000100:00100000:2.0:1788845039.661292:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641699456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.661307:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.661323:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae1e4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9808 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.661370:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660342912:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.661436:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.665137:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660342784:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845039.665164:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:1.0:1788845039.665183:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae1dd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9801 expref: 33 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.665203:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.665206:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.665212:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae1dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9801 expref: 33 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.665227:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845039.665298:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641699328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5282us (5591us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.665306:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1339 00000100:00100000:2.0:1788845039.665454:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660342912:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845039.665476:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae1e4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9808 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.665493:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae1e4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9808 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.665503:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845039.665527:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641699456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4238us (5754us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.665534:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 955 00000100:00100000:0.0:1788845039.669171:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.669212:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.669315:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641699584 00000100:00100000:2.0:1788845039.669327:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641699584 00000100:00100000:2.0:1788845039.669377:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1340 00000100:00100000:2.0:1788845039.669387:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641699584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.669409:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.669430:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae1eb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a980f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.669471:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.669512:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660343040:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.669595:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.670056:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641699712 00000100:00100000:1.0:1788845039.670063:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641699712 00000100:00100000:1.0:1788845039.670101:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 956 00000100:00100000:1.0:1788845039.670109:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641699712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.670120:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.670133:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ae1f2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9816 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.670170:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660343168:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.670221:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.675331:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660343168:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845039.675375:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ae1f2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9816 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.675402:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d7a0c9b/0xb2187301287ae1f2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9816 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.675415:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845039.675483:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641699712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5373us (6269us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.675492:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 956 00000100:00100000:0.0:1788845039.678682:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660343040:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.678712:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 724182 00010000:00010000:0.0:1788845039.678741:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae1eb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a980f expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.678763:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.678766:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.678772:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae1eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a980f expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.678788:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:0.0:1788845039.678847:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641699584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9462us (9685us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.678857:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1340 00000100:00100000:3.0:1788845039.682788:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.683175:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641699968 00000100:00100000:0.0:1788845039.683185:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641699968 00000100:00100000:0.0:1788845039.683430:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1341 00000100:00100000:0.0:1788845039.683442:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641699968:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845039.683498:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845039.683528:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae1ac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97d7 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.683552:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.683557:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845039.683964:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.683980:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 31 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.683988:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.684024:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 31 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.684042:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 31 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.684056:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660343296:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845039.684156:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.686925:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660343296:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845039.686954:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.686998:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.687771:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641700096 00000100:00100000:0.0:1788845039.687780:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641700096 00010000:00010000:0.0:1788845039.687818:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 32 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.687917:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 464 00000100:00100000:0.0:1788845039.687926:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641700096:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845039.687945:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845039.687951:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 32 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.687968:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 32 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.687989:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287ae1ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a97de expref: 32 pid: 5839 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.688005:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.688009:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.688021:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.688028:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.688055:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845039.688102:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+31:2370:x1875739641700096:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 177us (1104us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.688111:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 464 00010000:00010000:3.0:1788845039.689133:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.689143:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845039.689279:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967776 00000004:00010000:3.0:1788845039.689330:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 724182 -> 724182 00000004:00010000:3.0:1788845039.689334:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:3.0:1788845039.689337:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845036 00000004:00010000:3.0:1788845039.689339:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:3.0:1788845039.689340:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:3.0:1788845039.689347:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e9d39796 reply state 00000000f889bbd2 transno 4294967777 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.689359:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845039.689464:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:14891:x1875739641699968:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6022us (6683us total) trans 4294967777 rc 0/0 00000100:00100000:3.0:1788845039.689472:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1341 00000020:00080000:3.0:1788845039.689476:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845039.691631:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.691646:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.691653:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.691658:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845039.691663:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.691670:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.691687:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.691690:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845039.691694:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.691701:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae1f9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.693024:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.693071:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.693285:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641700224 00000100:00100000:3.0:1788845039.693295:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641700224 00002000:00010000:3.0:1788845039.693337:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845039.693343:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4700 with x1875739641700224 00000100:00100000:3.0:1788845039.693381:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 880 00000100:00100000:3.0:1788845039.693392:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641700224:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845039.693806:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 724182 -> 263778 00000004:00010000:3.0:1788845039.693810:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:3.0:1788845039.693813:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:3.0:1788845039.693816:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:3.0:1788845039.693818:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845039.693915:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2368:x1875739641700224:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 522us (895us total) trans 4294967778 rc 0/0 00000100:00100000:3.0:1788845039.693923:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 880 00002000:00010000:3.0:1788845039.693927:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845039.693930:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4700 with x1875739641700224 00000100:00100000:2.0:1788845039.694541:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641700352 00000100:00100000:2.0:1788845039.694548:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641700352 00002000:00010000:2.0:1788845039.694576:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845039.694580:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc4380 with x1875739641700352 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845039.694589:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc4380. 00000100:00100000:2.0:1788845039.694628:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 463 00000100:00100000:2.0:1788845039.694636:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641700352:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845039.694955:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1459157 -> 0 00002000:00010000:2.0:1788845039.694960:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00002000:00010000:2.0:1788845039.694963:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845039 -> 1788845038 00002000:00010000:2.0:1788845039.694965:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00002000:00010000:2.0:1788845039.694968:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:2.0:1788845039.695010:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641700352:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 373us (1936us total) trans 4294967465 rc 0/0 00000100:00100000:2.0:1788845039.695020:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 463 00002000:00010000:2.0:1788845039.695024:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845039.695026:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc4380 with x1875739641700352 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845039.695031:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc4380. 00000100:00100000:1.0:1788845039.699557:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.699694:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641700480 00000100:00100000:0.0:1788845039.699705:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641700480 00000100:00100000:0.0:1788845039.699807:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1342 00000100:00100000:0.0:1788845039.699819:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+31:14891:x1875739641700480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.699843:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.699863:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287ae200 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a981d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.699982:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.699998:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.700010:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845039.700037:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 263778 00000004:00200000:0.0:1788845039.700041:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845039.700058:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.700084:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287ae200 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 33 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.700100:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 32 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.700108:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.700110:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845039.700113:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006200, rc 0) 00000100:00100000:0.0:1788845039.700168:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:14891:x1875739641700480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 351us (619us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.700174:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1342 00000100:00100000:3.0:1788845039.702673:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.702714:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.702720:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641700608 00000100:00100000:1.0:1788845039.702740:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641700608 00000100:00100000:0.0:1788845039.702963:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641700736 00000100:00100000:0.0:1788845039.702967:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641700736 00000100:00100000:1.0:1788845039.703019:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1343 00000100:00100000:1.0:1788845039.703037:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+32:2369:x1875739641700608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.703059:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.703080:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae20e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9824 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.703120:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.703165:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660343424:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.703369:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.703451:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 957 00000100:00100000:0.0:1788845039.703462:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641700736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.703472:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.703485:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae215 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a982b expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.703524:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02380700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660343552:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.703565:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00002000:00010000:3.0:1788845039.705877:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845038 -> 1788845039 00000100:00100000:3.0:1788845039.705932:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02380700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660343552:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845039.705966:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae215 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a982b expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.705992:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae215 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a982b expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.706007:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:3.0:1788845039.706060:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641700736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2602us (3346us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.706069:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 957 00000004:00010000:1.0:1788845039.706694:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845039 00000100:00100000:1.0:1788845039.706717:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660343424:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845039.708232:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:1.0:1788845039.708712:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae20e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9824 expref: 33 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.708765:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.708769:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.708774:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae20e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9824 expref: 33 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.708790:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845039.708992:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+32:2369:x1875739641700608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5960us (6325us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.709002:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1343 00000100:00100000:0.0:1788845039.715204:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.715307:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.715406:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641700992 00000100:00100000:3.0:1788845039.715415:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641700992 00000100:00100000:3.0:1788845039.715477:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 958 00000100:00100000:3.0:1788845039.715488:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641700992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.715508:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.715527:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae21c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9839 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.715588:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035686a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660343680:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.715697:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641700864 00000100:00100000:2.0:1788845039.715701:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641700864 00000100:00100000:2.0:1788845039.715792:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1344 00000100:00100000:2.0:1788845039.715800:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641700864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.715810:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.715824:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae223 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9832 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.715854:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.715878:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660343808:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.715922:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.715923:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.718099:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035686a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660343680:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.718123:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660343808:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845039.718138:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:3.0:1788845039.718146:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae21c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9839 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.718149:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae223 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9832 expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.718170:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.718173:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287ae21c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9839 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.718173:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845039.718178:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae223 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9832 expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.718189:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00010000:00010000:2.0:1788845039.718204:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:3.0:1788845039.718243:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641700992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2756us (2992us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.718243:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641700864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2442us (3046us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.718252:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 958 00000100:00100000:2.0:1788845039.718252:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1344 00000100:00100000:1.0:1788845039.722791:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.724167:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641701120 00000100:00100000:0.0:1788845039.724182:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641701120 00002000:00100000:0.0:1788845039.724244:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657100 x1875739641701120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.724268:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657100 with x1875739641701120 00000100:00100000:0.0:1788845039.724339:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 881 00000100:00100000:0.0:1788845039.724349:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2368:x1875739641701120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845039.737147:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2368:x1875739641701120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12784us (14351us total) trans 0 rc 266240/266240 00000100:00100000:1.0:1788845039.737163:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 881 00002000:00100000:1.0:1788845039.737179:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657100 x1875739641701120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/440 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 266240/266240 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.737191:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657100 with x1875739641701120 00000100:00100000:1.0:1788845039.741388:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.741476:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.741551:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641701248 00000100:00100000:3.0:1788845039.741561:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641701248 00000100:00100000:3.0:1788845039.741630:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1345 00000100:00100000:3.0:1788845039.741641:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641701248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.741664:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.741684:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae22a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9840 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.741721:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.741776:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035686a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660343936:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.741870:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.743964:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035686a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660343936:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.743990:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:0.0:1788845039.744009:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae22a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9840 expref: 33 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.744030:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.744034:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.744040:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287ae22a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9840 expref: 33 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.744058:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:0.0:1788845039.744113:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+32:2368:x1875739641701248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2473us (2636us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.744121:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1345 00000100:00100000:2.0:1788845039.745642:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641701376 00000100:00100000:2.0:1788845039.745653:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641701376 00000100:00100000:0.0:1788845039.745986:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 959 00000100:00100000:0.0:1788845039.745997:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641701376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.746015:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.746034:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae231 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9847 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.746092:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03849f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660344064:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.746217:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.748557:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03849f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660344064:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845039.748597:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae231 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9847 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.748694:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae231 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9847 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.748708:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845039.748812:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641701376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2815us (7431us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.748824:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 959 00000100:00100000:1.0:1788845039.751704:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.751837:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.751949:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641701632 00000100:00100000:3.0:1788845039.751960:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641701632 00000100:00100000:3.0:1788845039.752023:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 960 00000100:00100000:3.0:1788845039.752034:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641701632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.752053:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.752070:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287ae238 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9855 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.752135:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035686a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660344192:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.752214:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.754574:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035686a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660344192:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845039.754623:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287ae238 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9855 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.754650:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287ae238 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9855 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.754664:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030600, rc 0) 00000100:00100000:1.0:1788845039.754771:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641701632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2738us (2933us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.754780:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 960 00000100:00100000:2.0:1788845039.756861:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641701504 00000100:00100000:2.0:1788845039.756875:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641701504 00000100:00100000:1.0:1788845039.757183:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1346 00000100:00100000:1.0:1788845039.757195:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641701504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.757215:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.757236:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae23f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a984e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.757346:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.757388:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660344320:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.757512:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.760280:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660344320:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845039.760303:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:1.0:1788845039.760322:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae23f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a984e expref: 33 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.760342:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.760345:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.760350:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae23f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a984e expref: 33 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.760363:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845039.760419:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+32:2370:x1875739641701504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3226us (8724us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.760427:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1346 00000100:00100000:3.0:1788845039.763431:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.763532:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641701760 00000100:00100000:0.0:1788845039.763543:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641701760 00000100:00100000:0.0:1788845039.763602:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1347 00000100:00100000:0.0:1788845039.763612:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:14891:x1875739641701760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.763633:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.763652:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae246 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a985c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.763753:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae24d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.763769:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae24d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.763780:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae24d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845039.763809:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 263778 00000004:00200000:0.0:1788845039.763814:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845039.763833:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae24d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.763858:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae246 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a985c expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.763873:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae24d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a985c expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.763939:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.763942:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845039.763944:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845039.764005:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641701760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 394us (583us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.764012:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1347 00000100:00100000:3.0:1788845039.768691:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.768809:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641701888 00000100:00100000:2.0:1788845039.768818:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641701888 00000100:00100000:2.0:1788845039.768917:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1348 00000100:00100000:2.0:1788845039.768929:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+32:14891:x1875739641701888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.768950:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.768973:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9863 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.769004:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.769016:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.769051:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.769070:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 19 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.769085:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660344448:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845039.769160:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.772799:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.772930:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641702016 00000100:00100000:0.0:1788845039.772940:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641702016 00002000:00100000:0.0:1788845039.772993:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f2d80 x1875739641702016/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.773005:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f2d80 with x1875739641702016 00010000:00010000:0.0:1788845039.773015:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 20 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.773036:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 20 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845039.773044:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f033f2d80 00000100:00100000:0.0:1788845039.773069:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 33 00000100:00100000:0.0:1788845039.773079:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641702016:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845039.773428:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660344448:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845039.773456:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.773471:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.773473:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845039.773478:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 33 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.773488:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845039.773519:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+33:14891:x1875739641701888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4593us (4838us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.773527:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1348 00000004:00010000:0.0:1788845039.777340:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 263778 -> 263778 00000004:00010000:0.0:1788845039.777348:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.777351:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845036 00000004:00010000:0.0:1788845039.777353:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.777356:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845039.777435:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2369:x1875739641702016:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4352us (4639us total) trans 4294967779 rc 0/0 00000100:00100000:0.0:1788845039.777446:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 33 00002000:00100000:0.0:1788845039.777459:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f2d80 x1875739641702016/t4294967779(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/448 e 0 to 0 dl 1788845050 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.777470:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f2d80 with x1875739641702016 00010000:00010000:0.0:1788845039.777479:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 22 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.777498:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 22 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845039.777506:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f033f2d80 00000100:00100000:3.0:1788845039.781053:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.781166:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641702144 00000100:00100000:2.0:1788845039.781177:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641702144 00010000:00010000:2.0:1788845039.781218:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 22 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.781271:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 465 00000100:00100000:2.0:1788845039.781281:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641702144:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845039.781297:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845039.781325:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 22 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.781345:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 22 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.781368:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae13c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9783 expref: 22 pid: 5841 timeout: 528 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.781385:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.781392:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 32 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.781403:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 32 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.781413:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 32 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.781452:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 32 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.781466:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660344576:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845039.781548:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.785232:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.785393:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660344576:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845039.785420:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845039.785480:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641702144:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4200us (4436us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.785495:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 465 00000100:00100000:0.0:1788845039.786040:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641702272 00000100:00100000:0.0:1788845039.786049:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641702272 00000100:00100000:0.0:1788845039.786108:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 882 00000100:00100000:0.0:1788845039.786118:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+33:2370:x1875739641702272:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845039.786373:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 263778 -> 263778 00000004:00010000:0.0:1788845039.786378:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.786381:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:0.0:1788845039.786383:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.786386:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:0.0:1788845039.786441:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+34:2370:x1875739641702272:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 324us (1218us total) trans 4294967780 rc 0/0 00000100:00100000:0.0:1788845039.786448:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 882 00000100:00100000:3.0:1788845039.793835:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.794119:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641702400 00000100:00100000:0.0:1788845039.794132:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641702400 00000100:00100000:0.0:1788845039.794205:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 961 00000100:00100000:0.0:1788845039.794215:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641702400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.794234:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.794262:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae25b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a986a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.794325:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03849f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660344704:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.794411:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.796642:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03849f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660344704:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845039.796686:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae25b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a986a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.796710:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae25b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a986a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.796722:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845039.796788:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641702400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2574us (2966us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.796797:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 961 00000100:00100000:3.0:1788845039.800065:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.800174:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641702528 00000100:00100000:1.0:1788845039.800185:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641702528 00000100:00100000:1.0:1788845039.800259:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 962 00000100:00100000:1.0:1788845039.800270:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641702528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.800288:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.800307:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae262 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9871 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.800363:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660344832:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.800442:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.803597:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660344832:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845039.803649:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae262 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9871 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.803674:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae262 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9871 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.803686:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845039.803752:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641702528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3483us (3696us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.803760:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 962 00000100:00100000:0.0:1788845039.807217:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.807409:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641702656 00000100:00100000:1.0:1788845039.807418:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641702656 00000100:00100000:1.0:1788845039.807486:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 883 00000100:00100000:1.0:1788845039.807496:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+34:2370:x1875739641702656:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845039.807762:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 263778 -> 263778 00000004:00010000:1.0:1788845039.807767:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.807769:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845036 -> 1788845036 00000004:00010000:1.0:1788845039.807772:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.807774:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845039.807849:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+35:2370:x1875739641702656:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 354us (641us total) trans 4294967781 rc 0/0 00000100:00100000:1.0:1788845039.807858:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 883 00000100:00100000:3.0:1788845039.817602:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.817734:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641702784 00000100:00100000:0.0:1788845039.817744:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641702784 00000100:00100000:0.0:1788845039.817808:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1349 00000100:00100000:0.0:1788845039.817819:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641702784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.817849:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.817914:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae269 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9878 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.817954:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 34 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.817996:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03849180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660344960:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.818074:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000004:00010000:0.0:1788845039.822129:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845036 -> 1788845039 00000100:00100000:0.0:1788845039.822162:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03849180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660344960:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.822184:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:0.0:1788845039.822204:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae269 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9878 expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.822226:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.822230:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.822235:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae269 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9878 expref: 22 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.822373:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845039.822438:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641702784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4621us (4846us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.822446:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1349 00000100:00100000:3.0:1788845039.824783:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.824911:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641702912 00000100:00100000:1.0:1788845039.824921:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641702912 00000100:00100000:1.0:1788845039.824979:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1350 00000100:00100000:1.0:1788845039.824989:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641702912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.825011:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.825032:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae270 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a987f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.825069:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 34 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.825112:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660345088:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.825190:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.827679:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660345088:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845039.827707:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:1.0:1788845039.827739:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae270 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a987f expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.827760:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.827764:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.827769:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae270 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a987f expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.827785:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845039.827844:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641702912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2856us (3069us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.827853:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1350 00000100:00100000:3.0:1788845039.831029:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.831136:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641703040 00000100:00100000:1.0:1788845039.831146:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641703040 00000100:00100000:1.0:1788845039.831197:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1351 00000100:00100000:1.0:1788845039.831208:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:14891:x1875739641703040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.831229:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.831264:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9886 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.831301:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 34 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.831312:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.831349:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 34 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.831369:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 34 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.831383:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660345216:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845039.831503:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.834975:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.835072:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641703168 00000100:00100000:0.0:1788845039.835082:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641703168 00010000:00010000:0.0:1788845039.835124:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 35 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.835289:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 466 00000100:00100000:0.0:1788845039.835300:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+35:2370:x1875739641703168:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845039.835317:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845039.835346:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 35 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.835372:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 35 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.835399:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.835406:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.835419:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.835428:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.835466:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.835483:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03849180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660345344:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845039.835558:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788845039.836754:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae254 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9863 expref: 35 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.836780:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660345216:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845039.836807:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.836821:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.836824:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.836827:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845039.836946:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:14891:x1875739641703040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5740us (5927us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845039.836955:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1351 00000100:00100000:3.0:1788845039.838861:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.838982:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641703296 00000100:00100000:1.0:1788845039.838991:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641703296 00000100:00100000:1.0:1788845039.839051:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 884 00000100:00100000:1.0:1788845039.839062:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641703296:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845039.839322:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 263778 -> 263778 00000004:00010000:1.0:1788845039.839326:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.839329:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.839331:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.839333:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845039.839399:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641703296:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 339us (543us total) trans 4294967782 rc 0/0 00000100:00100000:1.0:1788845039.839408:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 884 00000100:00100000:0.0:1788845039.840940:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03849180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660345344:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845039.840968:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845039.841019:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+34:2370:x1875739641703168:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5721us (6056us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845039.841041:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 466 00000100:00100000:3.0:1788845039.845791:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.846010:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641703424 00000100:00100000:1.0:1788845039.846020:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641703424 00000100:00100000:1.0:1788845039.846095:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 885 00000100:00100000:1.0:1788845039.846107:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641703424:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845039.846360:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 263778 -> 263778 00000004:00010000:1.0:1788845039.846364:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.846367:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.846369:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.846372:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845039.846448:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641703424:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 342us (666us total) trans 4294967783 rc 0/0 00000100:00100000:1.0:1788845039.846456:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 885 00000100:00100000:2.0:1788845039.849640:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.849689:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.849789:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641703552 00000100:00100000:0.0:1788845039.849799:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641703552 00000100:00100000:0.0:1788845039.849862:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1352 00000100:00100000:0.0:1788845039.849871:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+34:2370:x1875739641703552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.849921:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.849942:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae27e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a988d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.849980:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.850021:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03849180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660345472:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.850093:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.850536:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641703680 00000100:00100000:1.0:1788845039.850542:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641703680 00000100:00100000:1.0:1788845039.850585:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 963 00000100:00100000:1.0:1788845039.850594:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641703680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.850612:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.850624:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae285 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9894 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.850662:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660345600:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.850703:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.853979:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03849180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660345472:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.854005:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:0.0:1788845039.854026:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae27e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a988d expref: 35 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.854047:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.854051:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.854056:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae27e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a988d expref: 35 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.854078:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845039.854156:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+34:2370:x1875739641703552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4284us (4464us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.854164:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1352 00000100:00100000:1.0:1788845039.854824:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660345600:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845039.854847:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae285 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9894 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.854863:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287ae285 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9894 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.854873:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845039.854954:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641703680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4360us (5322us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.854962:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 963 00000100:00100000:2.0:1788845039.859153:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.859202:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.859280:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641703808 00000100:00100000:0.0:1788845039.859290:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641703808 00000100:00100000:0.0:1788845039.859348:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1353 00000100:00100000:0.0:1788845039.859359:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+34:2370:x1875739641703808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.859380:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.859399:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae28c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a989b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.859435:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.859472:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03849180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660345728:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.859552:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.860515:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641703936 00000100:00100000:3.0:1788845039.860523:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641703936 00000100:00100000:3.0:1788845039.860573:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 964 00000100:00100000:3.0:1788845039.860583:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641703936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.860596:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.860620:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae293 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98a2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.860663:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660345856:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.860739:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.868646:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660345856:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845039.868692:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae293 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a2 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.868715:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae293 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a2 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.868737:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:3.0:1788845039.868798:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641703936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8215us (9654us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.868807:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 964 00000100:00100000:0.0:1788845039.869547:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03849180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660345728:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.869565:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:0.0:1788845039.869580:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae28c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a989b expref: 35 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.869595:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.869598:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.869612:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae28c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a989b expref: 35 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.869627:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845039.869669:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+34:2370:x1875739641703808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10312us (10465us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.869676:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1353 00000100:00100000:2.0:1788845039.877404:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.877601:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641704064 00000100:00100000:1.0:1788845039.877620:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641704064 00002000:00100000:1.0:1788845039.877670:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be7b80 x1875739641704064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.877681:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be7b80 with x1875739641704064 00000100:00100000:1.0:1788845039.877718:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 886 00000100:00100000:1.0:1788845039.877738:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2369:x1875739641704064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845039.883995:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2369:x1875739641704064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 6257us (6592us total) trans 0 rc 266240/266240 00000100:00100000:1.0:1788845039.884009:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 886 00002000:00100000:1.0:1788845039.884022:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be7b80 x1875739641704064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:145/0 lens 488/440 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 266240/266240 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.884032:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be7b80 with x1875739641704064 00000100:00100000:3.0:1788845039.888794:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.888950:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641704192 00000100:00100000:2.0:1788845039.888962:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641704192 00000100:00100000:2.0:1788845039.889026:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1354 00000100:00100000:2.0:1788845039.889037:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+34:14891:x1875739641704192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.889058:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.889081:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98a9 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.889113:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.889122:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 35 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.889180:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.889198:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 23 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.889212:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660345984:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845039.889567:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91f540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.891708:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660345984:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845039.891750:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 35 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.891771:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.891774:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845039.891778:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 35 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.891786:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845039.891841:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+35:14891:x1875739641704192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2806us (3057us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845039.891851:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1354 00000100:00100000:1.0:1788845039.895099:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.895192:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641704320 00000100:00100000:3.0:1788845039.895202:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641704320 00010000:00010000:3.0:1788845039.895579:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 24 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.895635:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 467 00000100:00100000:3.0:1788845039.895647:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d11c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641704320:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845039.895661:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845039.895688:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 24 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.895706:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 24 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.895743:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae277 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9886 expref: 24 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.895759:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.895765:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 34 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.895777:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 34 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.895787:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 34 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.895830:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 34 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.895844:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660346112:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845039.896094:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f003c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.898849:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660346112:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845039.899376:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845039.899498:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d11c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641704320:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3853us (4406us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845039.899510:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 467 00000100:00100000:2.0:1788845039.904537:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.904578:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.904650:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641704448 00000100:00100000:0.0:1788845039.904660:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641704448 00002000:00100000:0.0:1788845039.904709:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0x10:0x0] req@ffff949f033f6d80 x1875739641704448/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.904721:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f6d80 with x1875739641704448 00000100:00100000:0.0:1788845039.904763:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641704576 00000100:00100000:0.0:1788845039.904767:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641704576 00002000:00100000:0.0:1788845039.904782:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f4e00 x1875739641704576/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845039.904790:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f4e00 with x1875739641704576 00000100:00100000:0.0:1788845039.904816:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 887 00000100:00100000:0.0:1788845039.904825:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+36:2369:x1875739641704448:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845039.905113:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 888 00000100:00100000:2.0:1788845039.905123:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+37:2370:x1875739641704576:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845039.905784:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+37:2370:x1875739641704576:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 660us (1201us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845039.905793:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 888 00002000:00100000:2.0:1788845039.905801:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f4e00 x1875739641704576/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/4536 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845039.905809:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f4e00 with x1875739641704576 00000100:00100000:1.0:1788845039.913076:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.913112:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641704704 00000100:00100000:1.0:1788845039.913123:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641704704 00002000:00100000:1.0:1788845039.913173:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367b80 x1875739641704704/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.913184:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367b80 with x1875739641704704 00000100:00100000:1.0:1788845039.913216:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 889 00000100:00100000:1.0:1788845039.913226:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+37:2370:x1875739641704704:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845039.913458:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+37:2370:x1875739641704704:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (391us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845039.913466:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 889 00002000:00100000:1.0:1788845039.913472:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02367b80 x1875739641704704/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/4536 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.913480:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02367b80 with x1875739641704704 00000100:00100000:2.0:1788845039.918972:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.919087:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641704832 00000100:00100000:3.0:1788845039.919098:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641704832 00000100:00100000:3.0:1788845039.919158:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1355 00000100:00100000:3.0:1788845039.919168:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641704832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.919190:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.919212:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae2a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98b0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.919391:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 36 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.919435:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660346240:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.919513:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008f00c00 going to sleep for 11 seconds 00000004:00010000:3.0:1788845039.921159:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb size from disk: 0 -> 57344 00000004:00010000:3.0:1788845039.921168:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb mtime from disk: 1788845003 -> 1788845037 00000004:00010000:3.0:1788845039.921171:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb atime from disk: 1788845003 -> 1788845003 00000004:00010000:3.0:1788845039.921174:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb ctime from disk: 1788845003 -> 1788845037 00000004:00010000:3.0:1788845039.921176:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb blocks from disk: 2 -> 2 00000100:00100000:3.0:1788845039.921423:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+37:2369:x1875739641704448:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 16594us (16892us total) trans 4294967784 rc 0/0 00000100:00100000:3.0:1788845039.921435:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 887 00002000:00100000:3.0:1788845039.921448:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0x10:0x0] req@ffff949f033f6d80 x1875739641704448/t4294967784(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 488/448 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.921458:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f6d80 with x1875739641704448 00000100:00100000:3.0:1788845039.925451:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660346240:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845039.925482:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:3.0:1788845039.925501:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae2a1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98b0 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.925522:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.925526:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845039.925531:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae2a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98b0 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.925547:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845039.925938:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641704832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6767us (6973us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.925949:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1355 00000100:00100000:2.0:1788845039.931519:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.931721:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641704960 00000100:00100000:1.0:1788845039.931740:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641704960 00000100:00100000:1.0:1788845039.931800:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1356 00000100:00100000:1.0:1788845039.931811:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641704960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845039.931832:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845039.931854:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae2a8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98b7 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.931920:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 36 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.931961:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660346368:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845039.932045:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.934989:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660346368:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845039.935021:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:1.0:1788845039.935040:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae2a8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98b7 expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.935063:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.935066:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.935071:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae2a8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98b7 expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.935085:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845039.935145:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641704960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3336us (3636us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845039.935153:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1356 00000100:00100000:2.0:1788845039.941168:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.941423:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641705088 00000100:00100000:3.0:1788845039.941434:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641705088 00002000:00100000:3.0:1788845039.941489:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f6a00 x1875739641705088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 504/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.941500:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f6a00 with x1875739641705088 00000100:00100000:3.0:1788845039.941534:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 890 00000100:00100000:3.0:1788845039.941544:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+37:2368:x1875739641705088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845039.947263:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+37:2368:x1875739641705088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 5699us (6083us total) trans 0 rc 253952/253952 00000100:00100000:3.0:1788845039.947278:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 890 00002000:00100000:3.0:1788845039.947293:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f6a00 x1875739641705088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 504/440 e 0 to 0 dl 1788845050 ref 1 fl Complete:/600/0 rc 253952/253952 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845039.947303:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f6a00 with x1875739641705088 00000100:00100000:1.0:1788845039.951379:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.951559:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641705216 00000100:00100000:2.0:1788845039.951570:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641705216 00000100:00100000:2.0:1788845039.951636:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1357 00000100:00100000:2.0:1788845039.951647:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641705216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.951670:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.951695:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae2af lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98be expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.951754:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 36 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.951801:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660346496:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.951938:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.953788:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660346496:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845039.953824:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:2.0:1788845039.953846:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae2af lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98be expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.953868:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.953871:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845039.953876:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae2af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98be expref: 24 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.953912:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845039.953966:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641705216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2321us (2597us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.953974:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1357 00000100:00100000:0.0:1788845039.957250:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.957340:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641705344 00000100:00100000:2.0:1788845039.957350:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641705344 00000100:00100000:2.0:1788845039.957410:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1358 00000100:00100000:2.0:1788845039.957421:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641705344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.957444:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.957465:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae2b6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98c5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.957499:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 36 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.957528:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660346624:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845039.957593:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845039.959876:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660346624:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845039.959949:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:2.0:1788845039.959968:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae2b6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98c5 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.959988:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845039.959990:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845039.959993:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae2b6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98c5 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.960004:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:2.0:1788845039.960045:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2367:x1875739641705344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2627us (2806us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845039.960052:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1358 00000100:00100000:1.0:1788845039.963120:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845039.963206:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641705472 00000100:00100000:3.0:1788845039.963218:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641705472 00000100:00100000:3.0:1788845039.963294:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1359 00000100:00100000:3.0:1788845039.963306:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641705472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845039.963327:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845039.963349:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae2bd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98cc expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.963390:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 36 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.963430:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660346752:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845039.963509:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a035f9f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845039.966263:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660346752:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845039.966292:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:3.0:1788845039.966312:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae2bd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98cc expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.966333:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.966336:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845039.966342:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae2bd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98cc expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.966356:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:3.0:1788845039.966418:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641705472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3114us (3307us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845039.966427:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1359 00000100:00100000:1.0:1788845039.969854:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.970091:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641705600 00000100:00100000:0.0:1788845039.970101:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641705600 00000100:00100000:0.0:1788845039.970158:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1360 00000100:00100000:0.0:1788845039.970167:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641705600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845039.970188:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845039.970209:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2c4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98d3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.970267:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 36 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.970310:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03848e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660346880:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845039.970391:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845039.972562:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03848e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660346880:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845039.972587:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 263778 00010000:00010000:0.0:1788845039.972617:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2c4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98d3 expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.972639:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.972642:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845039.972647:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98d3 expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.972665:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845039.972730:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2370:x1875739641705600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2554us (2876us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845039.972739:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1360 00000100:00100000:2.0:1788845039.980436:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.980541:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641705856 00000100:00100000:0.0:1788845039.980551:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641705856 00000100:00100000:0.0:1788845039.980611:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1361 00000100:00100000:0.0:1788845039.980621:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641705856:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845039.980674:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845039.980699:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae24d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a985c expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.980720:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.980734:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845039.980982:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.980994:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 36 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.981000:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.981024:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 36 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.981038:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 36 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.981048:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03848e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660347008:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845039.981117:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845039.984155:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845039.984170:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03848e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660347008:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845039.984197:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.984441:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641705984 00000100:00100000:3.0:1788845039.984448:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641705984 00010000:00010000:3.0:1788845039.984486:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 37 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.984524:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 468 00000100:00100000:3.0:1788845039.984534:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+37:2370:x1875739641705984:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845039.984549:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845039.984556:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 37 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.984572:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 37 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.984592:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae207 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a981d expref: 37 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.984679:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845039.984684:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.984694:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.984701:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845039.984739:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845039.984762:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.984769:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845039.984845:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+36:2370:x1875739641705984:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 312us (698us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845039.984853:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 468 00000001:00080000:0.0:1788845039.984917:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967785 00000004:00010000:0.0:1788845039.984970:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 263778 -> 263778 00000004:00010000:0.0:1788845039.984974:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845040 00000004:00010000:0.0:1788845039.984977:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845039.984979:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845040 00000004:00010000:0.0:1788845039.984981:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:0.0:1788845039.984989:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b15fab93 reply state 000000007dafe915 transno 4294967786 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845039.984997:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845039.985059:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641705856:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4440us (4633us total) trans 4294967786 rc 0/0 00000100:00100000:0.0:1788845039.985066:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1361 00000020:00080000:0.0:1788845039.985070:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788845039.991542:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845039.991771:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641706112 00000100:00100000:2.0:1788845039.991782:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641706112 00000100:00100000:2.0:1788845039.991852:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1362 00000100:00100000:2.0:1788845039.991865:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641706112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845039.991915:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845039.991938:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98da expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.991972:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 35 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.991981:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.992015:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 35 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845039.992033:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 35 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845039.992045:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660347136:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845039.992126:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00010000:00010000:0.0:1788845039.993463:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.993476:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.993483:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.993489:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845039.993494:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.993502:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.993523:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845039.993526:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845039.993530:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845039.993535:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae2cb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845039.994296:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660347136:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845039.994328:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.994346:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845039.994348:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845039.994353:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.994363:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032200, rc 0) 00000100:00100000:1.0:1788845039.994425:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641706112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2563us (2894us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845039.994434:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1362 00000100:00100000:2.0:1788845039.996937:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845039.997031:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641706240 00000100:00100000:1.0:1788845039.997044:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641706240 00002000:00100000:1.0:1788845039.997095:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f5f80 x1875739641706240/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 12776/0 e 0 to 0 dl 1788845050 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.997106:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f5f80 with x1875739641706240 00010000:00010000:1.0:1788845039.997115:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 36 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.997133:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 36 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845039.997140:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f033f5f80 00000100:00100000:1.0:1788845039.997176:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 34 00000100:00100000:1.0:1788845039.997189:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+36:2370:x1875739641706240:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845039.997560:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 263778 -> 263778 00000004:00010000:1.0:1788845039.997564:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845039 00000004:00010000:1.0:1788845039.997567:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845039.997569:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845039 00000004:00010000:1.0:1788845039.997572:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845039.997643:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+38:2370:x1875739641706240:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 457us (714us total) trans 4294967787 rc 0/0 00000100:00100000:1.0:1788845039.997652:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 34 00002000:00100000:1.0:1788845039.997660:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f5f80 x1875739641706240/t4294967787(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:145/0 lens 12776/448 e 0 to 0 dl 1788845050 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845039.997668:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f5f80 with x1875739641706240 00010000:00010000:1.0:1788845039.997673:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 38 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845039.997686:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 38 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845039.997692:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f033f5f80 00000100:00100000:3.0:1788845040.001317:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.001442:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641706368 00000100:00100000:2.0:1788845040.001452:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641706368 00010000:00010000:2.0:1788845040.001486:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 38 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.001519:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 469 00000100:00100000:2.0:1788845040.001525:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+38:2370:x1875739641706368:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845040.001536:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845040.001557:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 38 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.001570:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 38 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.001587:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033e00/0xb2187301287ae29a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98a9 expref: 38 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.001600:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.001613:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.001620:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.001627:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.001655:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.001668:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660347264:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845040.001753:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a01e91fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.003377:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660347264:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845040.003397:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845040.003435:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+37:2370:x1875739641706368:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 1910us (2131us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845040.003446:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 469 00000100:00100000:3.0:1788845040.004783:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.004874:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641706496 00000100:00100000:1.0:1788845040.004904:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641706496 00002000:00010000:1.0:1788845040.004943:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845040.004949:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f3480 with x1875739641706496 00000100:00100000:3.0:1788845040.004952:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.004986:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 891 00000100:00100000:1.0:1788845040.004996:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641706496:12345-192.168.201.29@tcp:10:fsx.0 00000100:00100000:0.0:1788845040.005058:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641706624 00000100:00100000:0.0:1788845040.005062:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641706624 00002000:00010000:0.0:1788845040.005082:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845040.005086:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f2680 with x1875739641706624 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845040.005096:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f2680. 00000100:00100000:0.0:1788845040.005112:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 464 00000100:00100000:0.0:1788845040.005118:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641706624:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845040.005193:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 263778 -> 600080 00000004:00010000:1.0:1788845040.005197:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845039 -> 1788845040 00000004:00010000:1.0:1788845040.005199:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845040.005201:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845039 -> 1788845040 00000004:00010000:1.0:1788845040.005203:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845040.005314:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:2370:x1875739641706496:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 318us (536us total) trans 4294967788 rc 0/0 00000100:00100000:1.0:1788845040.005322:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 891 00002000:00010000:1.0:1788845040.005326:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845040.005329:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f3480 with x1875739641706496 00002000:00010000:0.0:1788845040.005460:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1788845040.005464:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845039 -> 1788845040 00002000:00010000:0.0:1788845040.005466:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845039 -> 1788845038 00002000:00010000:0.0:1788845040.005468:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845039 -> 1788845040 00002000:00010000:0.0:1788845040.005471:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:0.0:1788845040.005494:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641706624:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 375us (541us total) trans 4294967466 rc 0/0 00000100:00100000:0.0:1788845040.005500:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 464 00002000:00010000:0.0:1788845040.005503:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845040.005505:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f2680 with x1875739641706624 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845040.005510:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f2680. 00000100:00100000:1.0:1788845040.010474:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.010587:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641706752 00000100:00100000:2.0:1788845040.010597:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641706752 00002000:00100000:2.0:1788845040.010647:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b800 x1875739641706752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.010658:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b800 with x1875739641706752 00000100:00100000:2.0:1788845040.010699:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 892 00000100:00100000:2.0:1788845040.010708:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:2368:x1875739641706752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.019000:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:2368:x1875739641706752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 8281us (8524us total) trans 0 rc 335872/335872 00000100:00100000:2.0:1788845040.019015:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 892 00002000:00100000:2.0:1788845040.019030:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9b800 x1875739641706752/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 335872/335872 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.019042:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9b800 with x1875739641706752 00000100:00100000:2.0:1788845040.028566:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.028779:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641706880 00000100:00100000:3.0:1788845040.028790:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641706880 00002000:00100000:3.0:1788845040.028844:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f7800 x1875739641706880/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.028856:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f7800 with x1875739641706880 00000100:00100000:3.0:1788845040.028934:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 893 00000100:00100000:3.0:1788845040.028945:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:2370:x1875739641706880:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845040.038195:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 600080 -> 600080 00000004:00010000:3.0:1788845040.038205:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:3.0:1788845040.038209:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:3.0:1788845040.038211:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:3.0:1788845040.038214:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845040.038418:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2370:x1875739641706880:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 9470us (9857us total) trans 4294967789 rc 0/0 00000100:00100000:3.0:1788845040.038429:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 893 00002000:00100000:3.0:1788845040.038443:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f7800 x1875739641706880/t4294967789(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/448 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.038453:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f7800 with x1875739641706880 00000100:00100000:2.0:1788845040.041459:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.041546:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641707008 00000100:00100000:3.0:1788845040.041557:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641707008 00000100:00100000:3.0:1788845040.041626:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1363 00000100:00100000:3.0:1788845040.041636:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+28:2370:x1875739641707008:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:2.0:1788845040.074078:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967786 is committed 00000001:00080000:2.0:1788845040.074096:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967780 is committed 00000001:00080000:2.0:1788845040.074127:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967788 is committed 00000001:00080000:2.0:1788845040.074133:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967787 is committed 00000001:00080000:2.0:1788845040.074174:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967789 is committed 00000100:00100000:1.0:1788845040.074518:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641707008:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 32877us (33064us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.074532:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1363 00000100:00100000:2.0:1788845040.078365:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.078513:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641707136 00000100:00100000:1.0:1788845040.078525:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641707136 00000100:00100000:1.0:1788845040.078585:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1364 00000100:00100000:1.0:1788845040.078596:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641707136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.078626:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.078648:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae2d9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98e1 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.078746:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.078760:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.078775:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845040.078802:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 600080 00000004:00200000:1.0:1788845040.078806:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845040.078823:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.078847:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae2d9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.078867:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.078874:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.078877:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845040.078879:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15e00, rc 0) 00000100:00100000:1.0:1788845040.078985:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641707136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 390us (628us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.078992:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1364 00000100:00100000:3.0:1788845040.084025:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.084166:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641707264 00000100:00100000:1.0:1788845040.084177:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641707264 00000100:00100000:1.0:1788845040.084235:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1365 00000100:00100000:1.0:1788845040.084246:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641707264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.084273:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.084297:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae2e7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98e8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.084395:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae2ee lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.084412:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae2ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.084425:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae2ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845040.084454:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 600080 00000004:00200000:1.0:1788845040.084459:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845040.084479:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae2ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.084505:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae2e7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e8 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.084522:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae2ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e8 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.084530:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.084533:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845040.084537:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845040.084610:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641707264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 366us (596us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.084618:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1365 00000100:00100000:0.0:1788845040.088274:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.088390:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641707392 00000100:00100000:1.0:1788845040.088401:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641707392 00000100:00100000:1.0:1788845040.088467:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1366 00000100:00100000:1.0:1788845040.088477:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a005695450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641707392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.088500:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.088521:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae2f5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98ef expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.088558:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.088615:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660347392:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.088697:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.091357:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.091444:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641707520 00000100:00100000:0.0:1788845040.091454:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641707520 00000100:00100000:0.0:1788845040.091512:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 965 00000100:00100000:0.0:1788845040.091522:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641707520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.091541:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.091559:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae2fc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98f6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.091630:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03848e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660347520:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.091711:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000004:00010000:1.0:1788845040.093373:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845039 -> 1788845040 00000100:00100000:1.0:1788845040.093405:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660347392:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845040.093434:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 600080 00010000:00010000:1.0:1788845040.093454:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae2f5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98ef expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.093474:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.093477:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845040.093482:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae2f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98ef expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.093494:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:1.0:1788845040.093553:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a005695450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641707392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5076us (5289us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.093562:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1366 00002000:00010000:0.0:1788845040.097700:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845038 -> 1788845040 00000100:00100000:0.0:1788845040.097746:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03848e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660347520:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845040.097785:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae2fc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98f6 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.097812:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae2fc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98f6 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.097825:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845040.097919:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641707520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6397us (6570us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.097928:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 965 00000100:00100000:2.0:1788845040.101444:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.101487:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.101581:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641707648 00000100:00100000:1.0:1788845040.101590:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641707648 00000100:00100000:1.0:1788845040.101666:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1367 00000100:00100000:1.0:1788845040.101678:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641707648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.101701:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.101732:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae303 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a98fd expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.101770:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.101821:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660347648:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.101940:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.102236:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641707776 00000100:00100000:3.0:1788845040.102240:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641707776 00000100:00100000:3.0:1788845040.102289:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 966 00000100:00100000:3.0:1788845040.102298:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641707776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.102309:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.102321:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae30a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9904 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.102358:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660347776:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845040.102650:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.106769:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660347776:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845040.106813:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae30a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9904 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.106841:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae30a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9904 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.106856:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:3.0:1788845040.106973:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641707776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4675us (5484us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845040.106982:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 966 00000100:00100000:1.0:1788845040.107574:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660347648:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845040.107592:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 600080 00010000:00010000:1.0:1788845040.107616:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae303 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98fd expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.107631:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.107634:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845040.107639:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae303 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98fd expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.107651:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:1.0:1788845040.107695:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641707648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6018us (6261us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.107703:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1367 00000100:00100000:2.0:1788845040.112456:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.112507:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.112596:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641708032 00000100:00100000:0.0:1788845040.112617:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641708032 00000100:00100000:0.0:1788845040.112678:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 967 00000100:00100000:0.0:1788845040.112689:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641708032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.112709:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.112757:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae311 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9912 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.112819:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03848e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660347904:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.112929:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.113103:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641707904 00000100:00100000:1.0:1788845040.113108:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641707904 00000100:00100000:1.0:1788845040.113139:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1368 00000100:00100000:1.0:1788845040.113146:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641707904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.113158:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.113173:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae318 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a990b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.113202:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.113229:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660348032:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.113274:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.116479:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03848e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660347904:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845040.116524:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae311 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9912 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.116551:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae311 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9912 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.116567:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:1.0:1788845040.116643:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641708032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3955us (4134us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.116653:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 967 00000100:00100000:2.0:1788845040.117586:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660348032:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845040.117622:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 600080 00010000:00010000:2.0:1788845040.117639:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae318 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a990b expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.117658:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.117661:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845040.117666:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15400/0xb2187301287ae318 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a990b expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.117680:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15400, rc 0) 00000100:00100000:2.0:1788845040.117746:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641707904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4600us (5301us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.117754:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1368 00000100:00100000:0.0:1788845040.121769:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.121819:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.121939:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641708288 00000100:00100000:1.0:1788845040.121951:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641708288 00000100:00100000:1.0:1788845040.122020:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 968 00000100:00100000:1.0:1788845040.122033:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641708288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.122057:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.122079:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ae31f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9920 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.122148:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660348160:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.122239:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.122428:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641708160 00000100:00100000:2.0:1788845040.122434:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641708160 00000100:00100000:2.0:1788845040.122475:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1369 00000100:00100000:2.0:1788845040.122484:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641708160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.122494:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.122508:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ae326 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9919 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.122536:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.122563:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660348288:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.122592:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.126285:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660348288:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845040.126318:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 600080 00010000:00010000:2.0:1788845040.126339:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ae326 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9919 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.126361:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.126364:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845040.126369:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ae326 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9919 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.126385:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032800, rc 0) 00000100:00100000:2.0:1788845040.126449:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641708160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3965us (4689us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.126459:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1369 00000100:00100000:1.0:1788845040.126954:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660348160:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845040.126985:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ae31f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9920 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.127003:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000131ed314/0xb2187301287ae31f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9920 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.127013:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:1.0:1788845040.127043:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641708288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5015us (5225us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.127050:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 968 00000100:00100000:0.0:1788845040.132198:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.132249:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.132336:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641708544 00000100:00100000:3.0:1788845040.132346:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641708544 00000100:00100000:3.0:1788845040.132410:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 969 00000100:00100000:3.0:1788845040.132421:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641708544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.132441:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.132461:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae32d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a992e expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.132524:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660348416:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845040.132620:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.133025:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641708416 00000100:00100000:1.0:1788845040.133032:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641708416 00000100:00100000:1.0:1788845040.133069:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1370 00000100:00100000:1.0:1788845040.133077:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641708416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.133091:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.133108:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ae334 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9927 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.133147:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.133178:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660348544:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.133229:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.136443:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660348416:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845040.136486:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae32d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a992e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.136513:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae32d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a992e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.136526:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:3.0:1788845040.136593:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641708544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4173us (4342us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845040.136610:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 969 00000100:00100000:1.0:1788845040.137141:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660348544:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845040.137165:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 600080 00010000:00010000:1.0:1788845040.137179:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ae334 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9927 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.137194:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.137197:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845040.137202:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ae334 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9927 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.137215:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845040.137262:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641708416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4185us (5075us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.137269:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1370 00000100:00100000:0.0:1788845040.145495:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.145576:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641708672 00000100:00100000:1.0:1788845040.145586:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641708672 00000100:00100000:1.0:1788845040.145663:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1371 00000100:00100000:1.0:1788845040.145674:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641708672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.145697:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.145733:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9935 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.145768:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.145779:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9935 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.145817:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.145836:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 10 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.145850:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660348672:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845040.145977:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.149189:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.149313:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660348672:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845040.149355:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9935 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.149374:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.149377:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845040.149382:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9935 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.149393:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:1.0:1788845040.149456:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641708672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3784us (3972us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.149463:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1371 00000100:00100000:0.0:1788845040.149650:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641708800 00000100:00100000:0.0:1788845040.149656:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641708800 00010000:00010000:0.0:1788845040.149680:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 11 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.149709:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 470 00000100:00100000:0.0:1788845040.149717:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641708800:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845040.149743:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845040.149767:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 11 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.149783:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 11 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.149803:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae2d2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98da expref: 11 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.149817:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.149822:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9935 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.149830:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9935 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.149839:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9935 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.149871:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9935 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.149936:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660348800:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845040.150056:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.153038:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660348800:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845040.153067:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845040.153169:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641708800:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3450us (3985us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.153181:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 470 00000100:00100000:2.0:1788845040.154567:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.154673:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641708928 00000100:00100000:3.0:1788845040.154681:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641708928 00002000:00100000:3.0:1788845040.154736:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f4700 x1875739641708928/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 504/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.154747:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f4700 with x1875739641708928 00000100:00100000:3.0:1788845040.154813:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 894 00000100:00100000:3.0:1788845040.154825:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641708928:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.160298:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641708928:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 5462us (5723us total) trans 0 rc 339968/339968 00000100:00100000:0.0:1788845040.160313:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 894 00002000:00100000:0.0:1788845040.160328:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f4700 x1875739641708928/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 504/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 339968/339968 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.160346:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f4700 with x1875739641708928 00000100:00100000:3.0:1788845040.163395:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.163999:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641709056 00000100:00100000:2.0:1788845040.164011:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641709056 00002000:00100000:2.0:1788845040.164066:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f1880 x1875739641709056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.164078:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f1880 with x1875739641709056 00000100:00100000:2.0:1788845040.164115:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 895 00000100:00100000:2.0:1788845040.164124:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641709056:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.164391:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641709056:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 267us (1005us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845040.164398:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 895 00002000:00100000:2.0:1788845040.164406:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f1880 x1875739641709056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.164414:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f1880 with x1875739641709056 00000100:00100000:3.0:1788845040.167435:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.167812:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641709184 00000100:00100000:2.0:1788845040.167823:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641709184 00002000:00100000:2.0:1788845040.167879:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f3100 x1875739641709184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.167926:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f3100 with x1875739641709184 00000100:00100000:2.0:1788845040.167962:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 896 00000100:00100000:2.0:1788845040.167972:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641709184:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.168380:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641709184:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 407us (953us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845040.168389:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 896 00002000:00100000:2.0:1788845040.168397:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f3100 x1875739641709184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.168406:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f3100 with x1875739641709184 00000100:00100000:3.0:1788845040.171993:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.172113:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641709312 00000100:00100000:0.0:1788845040.172123:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641709312 00002000:00100000:0.0:1788845040.172175:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f1f80 x1875739641709312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.172186:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f1f80 with x1875739641709312 00000100:00100000:0.0:1788845040.172224:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 897 00000100:00100000:0.0:1788845040.172235:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641709312:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.172548:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641709312:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 313us (565us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845040.172557:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 897 00002000:00100000:0.0:1788845040.172564:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f1f80 x1875739641709312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.172572:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f1f80 with x1875739641709312 00000100:00100000:3.0:1788845040.176138:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.176447:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641709440 00000100:00100000:2.0:1788845040.176458:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641709440 00000100:00100000:2.0:1788845040.176579:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1372 00000100:00100000:2.0:1788845040.176590:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641709440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.176622:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.176644:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a993c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.176682:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845040.176694:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 600080 00010000:00010000:2.0:1788845040.176702:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.176715:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.176718:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845040.176733:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:2.0:1788845040.176790:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641709440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 201us (659us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.176797:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1372 00000100:00100000:3.0:1788845040.179352:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.180057:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641709696 00000100:00100000:1.0:1788845040.180068:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641709696 00000100:00100000:1.0:1788845040.180121:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1373 00000100:00100000:1.0:1788845040.180131:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641709696:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845040.180183:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845040.180209:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae2ee lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e8 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.180227:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.180232:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845040.180508:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.180522:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.180530:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.180563:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.180581:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 10 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.181033:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660348928:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845040.181120:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.184129:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660348928:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845040.184162:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.184820:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.185036:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641709824 00000100:00100000:1.0:1788845040.185046:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641709824 00010000:00010000:1.0:1788845040.185087:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 11 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.185125:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:1.0:1788845040.185137:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641709824:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845040.185153:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845040.185161:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 11 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.185192:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 11 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.185213:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15e00/0xb2187301287ae2e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a98e1 expref: 11 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.185230:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.185234:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.185243:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.185251:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.185287:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845040.185346:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641709824:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 212us (529us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.185353:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 471 00010000:00010000:0.0:1788845040.185934:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.185944:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845040.186055:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967790 00000004:00010000:0.0:1788845040.186109:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 600080 -> 600080 00000004:00010000:0.0:1788845040.186112:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:0.0:1788845040.186115:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845040 -> 1788845039 00000004:00010000:0.0:1788845040.186116:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:0.0:1788845040.186119:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 648 00000004:00010000:0.0:1788845040.186125:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ebac1bb6 reply state 000000006e7a36dc transno 4294967791 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.186134:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845040.186219:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641709696:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6088us (6875us total) trans 4294967791 rc 0/0 00000100:00100000:0.0:1788845040.186227:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1373 00000020:00080000:0.0:1788845040.186231:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788845040.190114:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.190173:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641709952 00000100:00100000:3.0:1788845040.190181:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641709952 00000100:00100000:3.0:1788845040.190244:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1374 00000100:00100000:3.0:1788845040.190266:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641709952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.190291:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.190295:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845040.190341:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae33b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9935 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.190359:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.190364:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845040.190377:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9943 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.190402:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.190413:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.190454:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.190473:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 9 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.190484:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660349056:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845040.190572:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.194198:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:1.0:1788845040.194705:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.194728:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.194734:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.194740:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845040.194744:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.194753:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.194772:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.194775:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845040.194779:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.194784:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287ae349 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.194959:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641710080 00000100:00100000:3.0:1788845040.194971:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641710080 00010000:00010000:3.0:1788845040.195011:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 10 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.195055:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 472 00000100:00100000:3.0:1788845040.195065:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641710080:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845040.195079:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845040.195107:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 10 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.195123:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 10 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.195139:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.195145:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.195159:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.195167:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.195204:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.195219:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660349184:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845040.195452:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22840 going to sleep for 11 seconds 00010000:00010000:0.0:1788845040.197941:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033200/0xb2187301287ae342 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a993c expref: 10 pid: 5839 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.197964:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660349056:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845040.197990:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.198005:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.198007:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845040.198009:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:0.0:1788845040.202648:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641709952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12381us (12529us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.202660:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1374 00000100:00100000:3.0:1788845040.203966:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660349184:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845040.203997:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845040.204063:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641710080:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9000us (9875us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845040.204074:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 472 00000100:00100000:1.0:1788845040.206994:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.207095:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641710208 00000100:00100000:3.0:1788845040.207107:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641710208 00000100:00100000:3.0:1788845040.207173:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 970 00000100:00100000:3.0:1788845040.207183:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641710208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.207202:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.207225:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a994a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.207268:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.207278:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(5 14) 00010000:00010000:3.0:1788845040.207282:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 14) 00010000:00010000:3.0:1788845040.207287:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.207320:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.207337:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 9 pid: 7983 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.207351:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660349312:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845040.207432:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22840 going to sleep for 11 seconds 00010000:00010000:2.0:1788845040.210108:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845040.211181:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.211260:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641710336 00000100:00100000:1.0:1788845040.211269:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641710336 00010000:00010000:1.0:1788845040.211308:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 10 pid: 7983 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.211359:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 473 00000100:00100000:1.0:1788845040.211369:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641710336:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845040.211388:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845040.211411:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 10 pid: 7983 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.211430:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 10 pid: 7983 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.211455:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 14) 00010000:00010000:1.0:1788845040.211459:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 14) 00010000:00010000:1.0:1788845040.211469:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.211480:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.211520:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.211536:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660349440:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845040.211623:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00010000:00010000:3.0:1788845040.212051:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae143 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a978a expref: 10 pid: 7983 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.212069:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660349312:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845040.212094:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.212105:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:3.0:1788845040.212147:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641710208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4965us (5163us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845040.212159:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 970 00000100:00100000:1.0:1788845040.216793:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660349440:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845040.216828:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845040.216942:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641710336:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5573us (5767us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.216954:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 473 00000100:00100000:2.0:1788845040.220433:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.220518:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641710464 00000100:00100000:0.0:1788845040.220529:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641710464 00002000:00010000:0.0:1788845040.220570:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845040.220575:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f5880 with x1875739641710464 00000100:00100000:0.0:1788845040.220626:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 898 00000100:00100000:0.0:1788845040.220637:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641710464:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845040.221019:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 600080 -> 200622 00000004:00010000:0.0:1788845040.221023:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:0.0:1788845040.221026:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:0.0:1788845040.221028:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:0.0:1788845040.221030:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:0.0:1788845040.221099:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641710464:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 462us (677us total) trans 4294967792 rc 0/0 00000100:00100000:0.0:1788845040.221107:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 898 00002000:00010000:0.0:1788845040.221110:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845040.221113:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f5880 with x1875739641710464 00000100:00100000:2.0:1788845040.223487:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.223836:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641710592 00000100:00100000:0.0:1788845040.223847:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641710592 00002000:00010000:0.0:1788845040.223927:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845040.223933:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f5c00 with x1875739641710592 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845040.223945:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f5c00. 00000100:00100000:0.0:1788845040.223969:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 465 00000100:00100000:0.0:1788845040.223978:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641710592:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845040.224203:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1788845040.224206:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00002000:00010000:0.0:1788845040.224209:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845040 -> 1788845038 00002000:00010000:0.0:1788845040.224211:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00002000:00010000:0.0:1788845040.224213:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:0.0:1788845040.224449:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641710592:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 471us (971us total) trans 4294967467 rc 0/0 00000100:00100000:0.0:1788845040.224462:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 465 00002000:00010000:0.0:1788845040.224466:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845040.224468:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f5c00 with x1875739641710592 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845040.224473:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f5c00. 00000100:00100000:2.0:1788845040.227653:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.227743:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641710720 00000100:00100000:0.0:1788845040.227754:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641710720 00000100:00100000:0.0:1788845040.227860:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1375 00000100:00100000:0.0:1788845040.227872:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641710720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.227916:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.227935:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae35e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9951 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.228024:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.228038:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.228049:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845040.228076:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 200622 00000004:00200000:0.0:1788845040.228082:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845040.228100:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.228126:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae35e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.228146:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.228153:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.228156:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845040.228158:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845040.228219:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641710720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 348us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.228225:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1375 00000100:00100000:3.0:1788845040.231659:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.231807:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641710976 00000100:00100000:2.0:1788845040.231818:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641710976 00000100:00100000:2.0:1788845040.231874:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1376 00000100:00100000:2.0:1788845040.231912:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641710976:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845040.232184:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.232208:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.232216:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.232258:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.232276:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 17 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.232290:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f4380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660349568:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845040.232368:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.235842:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.235950:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641711104 00000100:00100000:0.0:1788845040.235961:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641711104 00010000:00010000:0.0:1788845040.236004:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 18 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.236046:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 474 00000100:00100000:0.0:1788845040.236056:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d13850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641711104:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845040.236072:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845040.236080:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 18 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.236098:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 18 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.236121:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.236126:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.236137:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.236145:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.236163:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845040.236212:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d13850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641711104:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 157us (381us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.236220:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 474 00010000:00010000:1.0:1788845040.236923:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae365 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9951 expref: 17 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.236942:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f4380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660349568:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845040.236962:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.236971:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.236977:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845040.237101:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967793 00000004:00010000:1.0:1788845040.237152:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 200622 -> 200622 00000004:00010000:1.0:1788845040.237156:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:1.0:1788845040.237158:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845040.237160:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:1.0:1788845040.237162:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000004:00010000:1.0:1788845040.237169:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000087e426b6 reply state 00000000c2688bad transno 4294967794 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.237176:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845040.237277:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641710976:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5394us (5627us total) trans 4294967794 rc 0/0 00000100:00100000:1.0:1788845040.237285:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1376 00000020:00080000:1.0:1788845040.237289:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845040.245267:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.245287:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.245295:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.245302:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02033600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845040.245308:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.245317:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.245336:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.245339:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845040.245343:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.245354:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae36c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.247586:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.247705:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641711232 00000100:00100000:1.0:1788845040.247716:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641711232 00000100:00100000:1.0:1788845040.247786:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1377 00000100:00100000:1.0:1788845040.247798:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641711232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.247825:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.247846:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9958 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.247880:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.247922:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.247959:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.247979:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.247993:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660349696:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845040.248118:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.251980:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.252565:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641711360 00000100:00100000:0.0:1788845040.252575:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641711360 00010000:00010000:0.0:1788845040.252628:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 17 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.252674:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 475 00000100:00100000:0.0:1788845040.252684:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641711360:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845040.252697:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845040.252736:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 17 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.252754:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 17 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.252777:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.252784:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.252796:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.252806:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.252850:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.252864:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660349824:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845040.252966:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.255619:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660349824:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845040.255647:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845040.255705:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641711360:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3022us (3734us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.255716:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 475 00010000:00010000:1.0:1788845040.257112:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae350 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9943 expref: 16 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.257139:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660349696:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845040.257174:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.257200:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.257203:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845040.257207:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845040.257443:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641711232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9646us (9866us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.257452:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1377 00000100:00100000:2.0:1788845040.260798:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.260977:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641711488 00000100:00100000:0.0:1788845040.260988:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641711488 00000100:00100000:0.0:1788845040.261125:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 971 00000100:00100000:0.0:1788845040.261136:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641711488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.261158:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.261175:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a995f expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.261205:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.261216:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 14) 00010000:00010000:0.0:1788845040.261220:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 14) 00010000:00010000:0.0:1788845040.261225:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.261276:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.261295:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 6 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.261311:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660349952:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845040.261393:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.264611:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.264683:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641711616 00000100:00100000:1.0:1788845040.264693:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641711616 00010000:00010000:1.0:1788845040.264741:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 7 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.264784:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 476 00000100:00100000:1.0:1788845040.264793:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641711616:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845040.264806:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845040.264828:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 7 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.264847:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 7 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.264873:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 14) 00010000:00010000:1.0:1788845040.264878:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 14) 00010000:00010000:1.0:1788845040.264914:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.264929:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.264965:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.264978:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660350080:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845040.265045:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00010000:00010000:0.0:1788845040.265426:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287ae357 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a994a expref: 7 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.265442:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660349952:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845040.265463:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.265474:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845040.265511:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641711488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4377us (4724us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.265518:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 971 00000100:00100000:2.0:1788845040.270044:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.270270:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660350080:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845040.270298:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845040.270371:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641711616:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5578us (5778us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.270382:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 476 00000100:00100000:3.0:1788845040.270580:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641711744 00000100:00100000:3.0:1788845040.270589:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641711744 00002000:00010000:3.0:1788845040.270636:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1500785->18446744073709551615) 00002000:00100000:3.0:1788845040.270640:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02382300 with x1875739641711744 ext(1500785->18446744073709551615) 00002000:00010000:3.0:1788845040.270651:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02382300. 00000100:00100000:3.0:1788845040.270681:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 466 00000100:00100000:3.0:1788845040.270691:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641711744:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845040.270959:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1500785 00002000:00010000:3.0:1788845040.270964:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00002000:00010000:3.0:1788845040.270966:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845038 -> 1788845038 00002000:00010000:3.0:1788845040.270969:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00002000:00010000:3.0:1788845040.270971:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:3.0:1788845040.271003:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641711744:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 312us (969us total) trans 4294967468 rc 0/0 00000100:00100000:3.0:1788845040.271010:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 466 00002000:00010000:3.0:1788845040.271013:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1500785->18446744073709551615) 00002000:00100000:3.0:1788845040.271016:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02382300 with x1875739641711744 ext(1500785->18446744073709551615) 00002000:00010000:3.0:1788845040.271021:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02382300. 00000100:00100000:2.0:1788845040.274503:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.274606:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641711872 00000100:00100000:0.0:1788845040.274615:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641711872 00000100:00100000:0.0:1788845040.274675:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1378 00000100:00100000:0.0:1788845040.274685:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641711872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.274714:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.274747:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae381 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9966 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.274844:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.274861:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.274871:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845040.274931:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1500785 00000004:00200000:0.0:1788845040.274935:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845040.274953:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.274978:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae381 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.274995:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.275002:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.275005:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845040.275008:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845040.275056:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641711872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 373us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.275062:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1378 00000100:00100000:2.0:1788845040.278540:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.278846:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641712000 00000100:00100000:3.0:1788845040.278857:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641712000 00000100:00100000:2.0:1788845040.279265:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1379 00000100:00100000:2.0:1788845040.279277:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641712000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.279300:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.279323:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032600/0xb2187301287ae38f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a996d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.279424:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae396 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.279439:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae396 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.279450:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae396 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845040.279476:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1500785 00000004:00200000:2.0:1788845040.279480:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845040.279501:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae396 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.279528:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032600/0xb2187301287ae38f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a996d expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.279549:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae396 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a996d expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.279558:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.279560:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845040.279564:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845040.279649:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641712000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 373us (1117us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.279655:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1379 00000100:00100000:3.0:1788845040.282591:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.284372:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.287972:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641712128 00000100:00100000:1.0:1788845040.287991:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641712128 00000100:00100000:1.0:1788845040.288027:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641712256 00000100:00100000:1.0:1788845040.288031:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641712256 00000100:00100000:1.0:1788845040.288156:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 972 00000100:00100000:1.0:1788845040.288166:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641712256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.288186:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.288206:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ae39d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a997b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.288279:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660350208:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.288364:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.288402:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1380 00000100:00100000:1.0:1788845040.288408:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641712128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.288417:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.288428:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae3a4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9974 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.288456:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.288478:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660350336:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.288501:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312180 going to sleep for 11 seconds 00002000:00010000:3.0:1788845040.293796:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845038 -> 1788845039 00000100:00100000:3.0:1788845040.293838:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660350208:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845040.293920:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ae39d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a997b expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.293946:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2111793/0xb2187301287ae39d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a997b expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.293962:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:3.0:1788845040.294045:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641712256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5879us (9680us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845040.294056:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 972 00000100:00100000:1.0:1788845040.295931:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660350336:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845040.295959:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 200622 00010000:00010000:1.0:1788845040.295977:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae3a4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9974 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.295997:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.296000:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845040.296005:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae3a4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9974 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.296020:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845040.296179:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641712128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7769us (13596us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.296188:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1380 00000100:00100000:2.0:1788845040.301037:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.301099:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.301154:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641712512 00000100:00100000:3.0:1788845040.301164:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641712512 00000100:00100000:3.0:1788845040.301219:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 973 00000100:00100000:3.0:1788845040.301228:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641712512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.301264:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.301282:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae3ab lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9989 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.301336:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660350464:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845040.301417:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.303912:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641712384 00000100:00100000:1.0:1788845040.303927:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641712384 00000100:00100000:1.0:1788845040.303984:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1381 00000100:00100000:1.0:1788845040.303994:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641712384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.304014:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.304036:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae3b2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9982 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.304075:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.304119:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660350592:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.304577:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.306602:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660350464:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845040.306648:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae3ab lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9989 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.306676:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae3ab lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9989 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.306694:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:0.0:1788845040.306780:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641712512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5552us (5680us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.306790:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 973 00000100:00100000:2.0:1788845040.312962:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660350592:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845040.312994:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 200622 00010000:00010000:2.0:1788845040.313014:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae3b2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9982 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.313036:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.313039:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845040.313045:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae3b2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9982 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.313060:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:2.0:1788845040.313125:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641712384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9131us (12096us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.313134:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1381 00000100:00100000:2.0:1788845040.318511:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.318558:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.318627:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641712640 00000100:00100000:0.0:1788845040.318638:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641712640 00000100:00100000:0.0:1788845040.318699:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1382 00000100:00100000:0.0:1788845040.318709:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641712640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.318745:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.318767:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae3b9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9990 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.318809:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.318853:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660350720:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.318971:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.319383:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641712768 00000100:00100000:3.0:1788845040.319389:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641712768 00000100:00100000:3.0:1788845040.319436:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 974 00000100:00100000:3.0:1788845040.319444:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641712768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.319456:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.319469:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae3c0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9997 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.319506:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660350848:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845040.319555:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.323487:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660350720:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845040.323517:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 200622 00010000:00010000:0.0:1788845040.323536:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae3b9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9990 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.323556:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.323559:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845040.323564:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae3b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9990 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.323579:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845040.323647:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641712640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4939us (5147us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.323656:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1382 00000100:00100000:3.0:1788845040.323825:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660350848:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845040.323853:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae3c0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9997 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.323869:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae3c0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9997 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.323925:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845040.323954:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641712768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4511us (5395us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845040.323961:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 974 00000100:00100000:2.0:1788845040.328500:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.328532:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.328631:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641713024 00000100:00100000:1.0:1788845040.328642:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641713024 00000100:00100000:1.0:1788845040.328704:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 975 00000100:00100000:1.0:1788845040.328715:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641713024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.328743:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.328763:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae3c7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99a5 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.328829:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660350976:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.328938:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.329841:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641712896 00000100:00100000:0.0:1788845040.329852:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641712896 00000100:00100000:0.0:1788845040.329930:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1383 00000100:00100000:0.0:1788845040.329939:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641712896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.329955:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.329971:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae3ce lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a999e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.330005:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.330040:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384a680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660351104:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.330099:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.333936:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384a680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660351104:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845040.333966:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 200622 00010000:00010000:0.0:1788845040.333986:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae3ce lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a999e expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.334008:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.334011:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845040.334016:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287ae3ce lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a999e expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.334033:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845040.334086:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641712896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4147us (5596us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.334094:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1383 00000100:00100000:1.0:1788845040.335005:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660350976:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845040.335034:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae3c7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99a5 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.335059:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae3c7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99a5 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.335070:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845040.335122:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641713024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6409us (6589us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.335130:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 975 00000100:00100000:2.0:1788845040.348500:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.348575:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641713152 00000100:00100000:0.0:1788845040.348584:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641713152 00002000:00100000:0.0:1788845040.348642:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383800 x1875739641713152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.348652:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383800 with x1875739641713152 00000100:00100000:0.0:1788845040.348691:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 899 00000100:00100000:0.0:1788845040.348700:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641713152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.351972:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.352069:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641713280 00000100:00100000:1.0:1788845040.352078:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641713280 00002000:00100000:1.0:1788845040.352125:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1503231) req@ffff949f02380700 x1875739641713280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.352136:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02380700 with x1875739641713280 ext(1048576->1503231) 00002000:00010000:1.0:1788845040.352141:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02380700 00000100:00100000:1.0:1788845040.352172:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 467 00000100:00100000:1.0:1788845040.352181:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641713280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.353619:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641713152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4910us (5120us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.353631:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 899 00002000:00100000:0.0:1788845040.353641:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02383800 x1875739641713152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.353650:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02383800 with x1875739641713152 00000100:00100000:2.0:1788845040.362691:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.362747:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.362876:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641713536 00000100:00100000:0.0:1788845040.362918:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641713536 00000100:00100000:0.0:1788845040.362977:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 976 00000100:00100000:0.0:1788845040.362988:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641713536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.363006:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.363024:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae3d5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99b3 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.363083:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660351232:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.363164:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.363286:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641713408 00000100:00100000:1.0:1788845040.363291:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641713408 00000100:00100000:1.0:1788845040.363331:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1384 00000100:00100000:1.0:1788845040.363341:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641713408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.363351:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.363366:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae3dc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99ac expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.363402:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.363427:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660351360:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.363454:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.367547:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641713280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 15356us (15573us total) trans 0 rc 454656/454656 00000100:00100000:1.0:1788845040.367563:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 467 00002000:00100000:1.0:1788845040.367576:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1503231) req@ffff949f02380700 x1875739641713280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 454656/454656 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.367587:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02380700 with x1875739641713280 ext(1048576->1503231) 00002000:00010000:1.0:1788845040.367594:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02380700 00000100:00100000:0.0:1788845040.371588:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660351232:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845040.371637:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae3d5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99b3 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.371662:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287ae3d5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99b3 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.371675:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845040.371738:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641713536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8752us (8990us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.371747:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 976 00000004:00010000:1.0:1788845040.371960:0:11036:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 200622 -> 764165 00000100:00100000:1.0:1788845040.371980:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660351360:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845040.371996:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 764165 00010000:00010000:1.0:1788845040.372007:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae3dc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99ac expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.372020:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845040.372024:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845040.372027:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae3dc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99ac expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.372038:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845040.372060:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641713408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8722us (9380us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.372067:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1384 00000100:00100000:0.0:1788845040.376442:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.376527:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.376607:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641713792 00000100:00100000:3.0:1788845040.376617:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641713792 00000100:00100000:3.0:1788845040.376677:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 977 00000100:00100000:3.0:1788845040.376687:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641713792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.376707:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.376736:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae3e3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99c1 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.376798:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660351488:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845040.376911:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.379805:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660351488:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845040.379848:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae3e3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c1 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.379872:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae3e3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c1 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.379929:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:0.0:1788845040.379988:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641713792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3302us (3460us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.379997:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 977 00000100:00100000:2.0:1788845040.381693:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641713664 00000100:00100000:2.0:1788845040.381707:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641713664 00000100:00100000:2.0:1788845040.381823:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1385 00000100:00100000:2.0:1788845040.381833:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641713664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.381853:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.381874:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3ea lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99ba expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.381936:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.381979:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660351616:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.382074:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.385401:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660351616:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845040.385430:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 764165 00010000:00010000:2.0:1788845040.385449:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3ea lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99ba expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.385469:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.385472:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845040.385477:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3ea lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99ba expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.385490:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:2.0:1788845040.385578:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641713664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3747us (9347us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.385587:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1385 00000100:00100000:0.0:1788845040.390626:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.390752:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641714048 00000100:00100000:2.0:1788845040.390763:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641714048 00000100:00100000:2.0:1788845040.390826:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1386 00000100:00100000:2.0:1788845040.390836:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641714048:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845040.390927:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845040.390955:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae396 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a996d expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.390972:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.390977:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845040.391171:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.391186:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.391194:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.391224:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.391262:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 12 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.391276:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660351744:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845040.391360:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.394452:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.394641:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641714176 00000100:00100000:3.0:1788845040.394651:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641714176 00010000:00010000:3.0:1788845040.394691:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 13 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.394741:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 477 00000100:00100000:3.0:1788845040.394752:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641714176:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845040.394766:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845040.394774:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 13 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.394791:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 13 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.394813:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.394818:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.394829:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.394837:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.394856:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845040.395160:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641714176:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 408us (716us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.395168:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 477 00010000:00010000:2.0:1788845040.395741:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae388 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9966 expref: 12 pid: 5841 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.395761:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660351744:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845040.395782:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.395795:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.395801:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845040.395941:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967795 00000004:00010000:2.0:1788845040.395992:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 764165 -> 200622 00000004:00010000:2.0:1788845040.395996:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:2.0:1788845040.395999:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:2.0:1788845040.396001:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:2.0:1788845040.396003:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000004:00010000:2.0:1788845040.396010:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000033c7729b reply state 00000000e3423902 transno 4294967796 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.396019:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845040.396107:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641714048:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5273us (5489us total) trans 4294967796 rc 0/0 00000100:00100000:2.0:1788845040.396115:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1386 00000020:00080000:2.0:1788845040.396118:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845040.399295:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.399323:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.399332:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.399339:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02033c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845040.399344:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.399352:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.399370:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.399373:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845040.399377:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.399385:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae3f1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.400162:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.400792:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641714304 00000100:00100000:2.0:1788845040.400803:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641714304 00000100:00100000:2.0:1788845040.400952:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1387 00000100:00100000:2.0:1788845040.400963:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641714304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.400990:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.401012:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99c8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.401047:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.401056:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.401093:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.401111:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 11 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.401125:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660351872:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845040.401210:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.405326:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660351872:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845040.405365:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.405384:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.405387:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845040.405390:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.405399:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845040.405447:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641714304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4486us (5291us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.405456:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1387 00000100:00100000:3.0:1788845040.407415:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.407488:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641714432 00000100:00100000:0.0:1788845040.407497:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641714432 00002000:00100000:0.0:1788845040.407547:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f0380 x1875739641714432/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.407558:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f0380 with x1875739641714432 00010000:00010000:0.0:1788845040.407566:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.407585:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 12 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845040.407602:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f033f0380 00000100:00100000:0.0:1788845040.407630:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 35 00000100:00100000:0.0:1788845040.407638:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641714432:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845040.411133:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 200622 -> 764165 00000004:00010000:1.0:1788845040.411141:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:1.0:1788845040.411143:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:1.0:1788845040.411145:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:1.0:1788845040.411148:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:1.0:1788845040.411223:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641714432:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3580us (3811us total) trans 4294967797 rc 0/0 00000100:00100000:1.0:1788845040.411233:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 35 00002000:00100000:1.0:1788845040.411396:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f033f0380 x1875739641714432/t4294967797(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/448 e 0 to 0 dl 1788845051 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.411405:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f0380 with x1875739641714432 00010000:00010000:1.0:1788845040.411415:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 14 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.411435:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 14 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845040.411444:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f033f0380 00000100:00100000:0.0:1788845040.415109:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.415190:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641714560 00000100:00100000:3.0:1788845040.415200:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641714560 00010000:00010000:3.0:1788845040.415259:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 14 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.415303:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:3.0:1788845040.415313:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641714560:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845040.415326:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845040.415352:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 14 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.415371:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 14 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.415394:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae373 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9958 expref: 14 pid: 11036 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.415410:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.415417:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.415428:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.415437:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.415473:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.415493:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660352000:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845040.415577:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.419213:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660352000:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845040.419253:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845040.419311:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641714560:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3999us (4212us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.419322:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:0.0:1788845040.421581:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.421668:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641714688 00000100:00100000:3.0:1788845040.421679:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641714688 00000100:00100000:3.0:1788845040.421747:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 978 00000100:00100000:3.0:1788845040.421758:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641714688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.421777:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.421796:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99cf expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.421833:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.421843:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 14) 00010000:00010000:3.0:1788845040.421847:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 14) 00010000:00010000:3.0:1788845040.421853:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.421925:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.421945:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 10 pid: 6682 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.421965:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660352128:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845040.422050:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22480 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.425734:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.425928:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641714816 00000100:00100000:0.0:1788845040.425939:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641714816 00010000:00010000:0.0:1788845040.425986:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 11 pid: 6682 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.426027:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 479 00000100:00100000:0.0:1788845040.426037:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641714816:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845040.426053:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845040.426080:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 11 pid: 6682 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.426099:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 11 pid: 6682 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.426124:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 14) 00010000:00010000:0.0:1788845040.426129:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 14) 00010000:00010000:0.0:1788845040.426137:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.426147:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.426185:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.426200:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660352256:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845040.426297:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00010000:00010000:1.0:1788845040.427522:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287ae37a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a995f expref: 11 pid: 6682 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.427546:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660352128:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845040.427578:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.427604:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:1.0:1788845040.427672:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641714688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5915us (6102us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.427680:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 978 00000100:00100000:1.0:1788845040.431120:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660352256:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845040.431147:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845040.431204:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641714816:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5168us (5489us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.431214:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 479 00000100:00100000:1.0:1788845040.432112:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.432152:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.432261:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641714944 00000100:00100000:3.0:1788845040.432270:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641714944 00002000:00010000:3.0:1788845040.432309:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845040.432314:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9aa00 with x1875739641714944 00000100:00100000:3.0:1788845040.432340:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 900 00000100:00100000:3.0:1788845040.432350:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641714944:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845040.432638:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 764165 -> 411260 00000004:00010000:3.0:1788845040.432642:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:3.0:1788845040.432644:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:3.0:1788845040.432647:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:3.0:1788845040.432649:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:3.0:1788845040.432699:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641714944:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 351us (590us total) trans 4294967798 rc 0/0 00000100:00100000:3.0:1788845040.432706:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 900 00002000:00010000:3.0:1788845040.432710:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845040.432712:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9aa00 with x1875739641714944 00000100:00100000:2.0:1788845040.435429:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641715072 00000100:00100000:2.0:1788845040.435444:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641715072 00002000:00010000:2.0:1788845040.435489:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845040.435495:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01f98a80 with x1875739641715072 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845040.435508:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01f98a80. 00000100:00100000:2.0:1788845040.435615:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 468 00000100:00100000:2.0:1788845040.435627:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641715072:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845040.435953:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1500785 -> 0 00002000:00010000:2.0:1788845040.435957:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00002000:00010000:2.0:1788845040.435960:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845039 -> 1788845038 00002000:00010000:2.0:1788845040.435962:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00002000:00010000:2.0:1788845040.435964:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:2.0:1788845040.436050:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641715072:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 425us (3897us total) trans 4294967469 rc 0/0 00000100:00100000:2.0:1788845040.436059:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 468 00002000:00010000:2.0:1788845040.436063:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845040.436066:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01f98a80 with x1875739641715072 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845040.436071:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01f98a80. 00000100:00100000:1.0:1788845040.438940:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.439182:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641715200 00000100:00100000:3.0:1788845040.439192:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641715200 00000100:00100000:3.0:1788845040.439267:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1388 00000100:00100000:3.0:1788845040.439278:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641715200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.439304:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.439325:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae406 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99d6 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.439414:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae40d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.439429:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae40d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.439439:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae40d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845040.439466:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 411260 00000004:00200000:3.0:1788845040.439471:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845040.439487:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae40d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.439514:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae406 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99d6 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.439532:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae40d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99d6 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.439541:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.439543:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845040.439547:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031400, rc 0) 00000100:00100000:3.0:1788845040.439617:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641715200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 341us (686us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845040.439624:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1388 00000100:00100000:3.0:1788845040.443792:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.443839:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.443923:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641715456 00000100:00100000:0.0:1788845040.443933:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641715456 00000100:00100000:0.0:1788845040.443979:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 979 00000100:00100000:0.0:1788845040.443988:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641715456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.444005:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.444022:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae414 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99e4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.444075:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660352384:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.444147:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.444613:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641715328 00000100:00100000:2.0:1788845040.444619:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641715328 00000100:00100000:2.0:1788845040.444651:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1389 00000100:00100000:2.0:1788845040.444659:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641715328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.444673:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.444687:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae41b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99dd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.444726:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.444752:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660352512:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.444800:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00002000:00010000:0.0:1788845040.448568:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845038 -> 1788845039 00000100:00100000:0.0:1788845040.448611:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660352384:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845040.448650:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae414 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99e4 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.448673:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae414 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99e4 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.448686:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006800, rc 0) 00000100:00100000:0.0:1788845040.448755:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641715456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4766us (4914us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.448765:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 979 00000100:00100000:2.0:1788845040.449406:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660352512:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845040.449423:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:2.0:1788845040.449439:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae41b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99dd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.449453:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.449456:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845040.449460:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae41b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99dd expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.449472:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845040.449517:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641715328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4859us (5735us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.449524:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1389 00000100:00100000:3.0:1788845040.454567:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.454790:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641715584 00000100:00100000:1.0:1788845040.454801:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641715584 00000100:00100000:1.0:1788845040.454866:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1390 00000100:00100000:1.0:1788845040.454875:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641715584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.454941:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.454966:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99eb expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.455009:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.455018:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.455053:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.455073:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 20 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.455087:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660352640:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845040.455177:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.459085:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.459184:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641715712 00000100:00100000:0.0:1788845040.459195:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641715712 00010000:00010000:0.0:1788845040.459258:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 21 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.459304:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 480 00000100:00100000:0.0:1788845040.459315:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641715712:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845040.459330:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845040.459360:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 21 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.459380:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 21 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.459404:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.459411:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.459423:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.459433:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.459477:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.459493:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660352768:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845040.459599:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00010000:00010000:3.0:1788845040.460164:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae3f8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99c8 expref: 21 pid: 5840 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.460183:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660352640:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845040.460209:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.460222:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.460224:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845040.460227:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:3.0:1788845040.460334:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641715584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5459us (5775us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845040.460343:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1390 00000100:00100000:0.0:1788845040.464517:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660352768:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845040.464544:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845040.464614:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641715712:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5300us (5539us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.464627:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 480 00000100:00100000:3.0:1788845040.466209:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.466423:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641715840 00000100:00100000:1.0:1788845040.466435:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641715840 00002000:00100000:1.0:1788845040.466486:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684700 x1875739641715840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.466497:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684700 with x1875739641715840 00000100:00100000:1.0:1788845040.466538:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 901 00000100:00100000:1.0:1788845040.466547:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641715840:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.466776:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641715840:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 230us (577us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845040.466785:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 901 00002000:00100000:1.0:1788845040.466792:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035684700 x1875739641715840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.466800:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035684700 with x1875739641715840 00000100:00100000:2.0:1788845040.471600:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.471678:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641715968 00000100:00100000:3.0:1788845040.471688:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641715968 00002000:00100000:3.0:1788845040.471745:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f655500 x1875739641715968/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.471755:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f655500 with x1875739641715968 00000100:00100000:3.0:1788845040.471790:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 902 00000100:00100000:3.0:1788845040.471799:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641715968:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845040.472031:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641715968:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 232us (444us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845040.472038:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 902 00002000:00100000:3.0:1788845040.472045:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f655500 x1875739641715968/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.472053:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f655500 with x1875739641715968 00000100:00100000:3.0:1788845040.475400:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.475515:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641716096 00000100:00100000:1.0:1788845040.475526:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641716096 00002000:00100000:1.0:1788845040.475578:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685180 x1875739641716096/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.475589:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685180 with x1875739641716096 00000100:00100000:1.0:1788845040.475639:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 903 00000100:00100000:1.0:1788845040.475648:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641716096:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.475943:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641716096:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 295us (551us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845040.475951:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 903 00002000:00100000:1.0:1788845040.475959:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685180 x1875739641716096/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.475967:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685180 with x1875739641716096 00000100:00100000:1.0:1788845040.480034:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.480175:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641716224 00000100:00100000:3.0:1788845040.480188:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641716224 00002000:00100000:3.0:1788845040.480257:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9bb80 x1875739641716224/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.480269:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9bb80 with x1875739641716224 00000100:00100000:3.0:1788845040.480309:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 904 00000100:00100000:3.0:1788845040.480320:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641716224:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845040.480561:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641716224:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (538us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845040.480568:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 904 00002000:00100000:3.0:1788845040.480575:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9bb80 x1875739641716224/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.480582:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9bb80 with x1875739641716224 00000100:00100000:2.0:1788845040.484124:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.484230:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641716352 00000100:00100000:1.0:1788845040.484361:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641716352 00002000:00100000:1.0:1788845040.484420:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656680 x1875739641716352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.484431:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656680 with x1875739641716352 00000100:00100000:1.0:1788845040.484474:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 905 00000100:00100000:1.0:1788845040.484486:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641716352:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.484868:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641716352:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 383us (750us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845040.484877:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 905 00002000:00100000:1.0:1788845040.485489:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656680 x1875739641716352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.485500:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656680 with x1875739641716352 00000100:00100000:1.0:1788845040.488403:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.488489:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641716480 00000100:00100000:3.0:1788845040.488499:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641716480 00002000:00100000:3.0:1788845040.488549:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f98380 x1875739641716480/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.488560:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98380 with x1875739641716480 00000100:00100000:3.0:1788845040.488602:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 906 00000100:00100000:3.0:1788845040.488611:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641716480:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845040.488919:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641716480:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 308us (524us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845040.488927:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 906 00002000:00100000:3.0:1788845040.488934:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f98380 x1875739641716480/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.488941:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f98380 with x1875739641716480 00000100:00100000:3.0:1788845040.492665:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.492796:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641716608 00000100:00100000:0.0:1788845040.492807:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641716608 00002000:00100000:0.0:1788845040.492857:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685880 x1875739641716608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.492867:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685880 with x1875739641716608 00000100:00100000:0.0:1788845040.492988:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 907 00000100:00100000:0.0:1788845040.492998:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641716608:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.493228:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641716608:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (573us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845040.493247:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 907 00002000:00100000:0.0:1788845040.493255:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035685880 x1875739641716608/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.493263:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035685880 with x1875739641716608 00000100:00100000:2.0:1788845040.496493:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.497190:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641716736 00000100:00100000:1.0:1788845040.497200:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641716736 00002000:00100000:1.0:1788845040.497388:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656300 x1875739641716736/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.497400:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656300 with x1875739641716736 00000100:00100000:1.0:1788845040.497440:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 908 00000100:00100000:1.0:1788845040.497450:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641716736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.497801:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641716736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 352us (1315us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845040.497808:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 908 00002000:00100000:1.0:1788845040.497815:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656300 x1875739641716736/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.497822:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656300 with x1875739641716736 00000100:00100000:3.0:1788845040.500928:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.501055:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641716864 00000100:00100000:2.0:1788845040.501066:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641716864 00002000:00100000:2.0:1788845040.501117:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686680 x1875739641716864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.501133:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686680 with x1875739641716864 00000100:00100000:2.0:1788845040.501170:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 909 00000100:00100000:2.0:1788845040.501181:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641716864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.501433:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641716864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (515us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845040.501441:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 909 00002000:00100000:2.0:1788845040.501447:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686680 x1875739641716864/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.501455:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686680 with x1875739641716864 00000100:00100000:3.0:1788845040.506741:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.506851:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641716992 00000100:00100000:1.0:1788845040.506862:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641716992 00002000:00100000:1.0:1788845040.506952:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686a00 x1875739641716992/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.506962:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686a00 with x1875739641716992 00000100:00100000:1.0:1788845040.506991:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 910 00000100:00100000:1.0:1788845040.507001:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641716992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.507213:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641716992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 213us (480us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845040.507221:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 910 00002000:00100000:1.0:1788845040.507228:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035686a00 x1875739641716992/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.507416:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035686a00 with x1875739641716992 00000100:00100000:0.0:1788845040.510679:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.510927:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641717120 00000100:00100000:2.0:1788845040.510937:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641717120 00002000:00100000:2.0:1788845040.510984:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7d500 x1875739641717120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.510995:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7d500 with x1875739641717120 00000100:00100000:2.0:1788845040.511030:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 911 00000100:00100000:2.0:1788845040.511040:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641717120:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.511458:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641717120:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 417us (786us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845040.511466:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 911 00002000:00100000:2.0:1788845040.511473:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7d500 x1875739641717120/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.511481:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7d500 with x1875739641717120 00000100:00100000:1.0:1788845040.515207:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.515422:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641717248 00000100:00100000:0.0:1788845040.515433:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641717248 00000100:00100000:0.0:1788845040.515547:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1391 00000100:00100000:0.0:1788845040.515559:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641717248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.515578:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.515611:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae429 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99f2 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.515712:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.515735:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.515747:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845040.515776:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 411260 00000004:00200000:0.0:1788845040.515780:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845040.515800:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.515827:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae429 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.515844:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.515854:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.515856:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845040.515860:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:0.0:1788845040.516002:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641717248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 445us (803us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.516009:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1391 00000100:00100000:3.0:1788845040.519113:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.519153:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641717376 00000100:00100000:3.0:1788845040.519161:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641717376 00000100:00100000:3.0:1788845040.519215:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 980 00000100:00100000:3.0:1788845040.519226:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641717376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.519259:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.519277:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae437 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a99f9 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.519335:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660352896:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845040.519412:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.521875:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660352896:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.521971:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae437 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f9 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.521994:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287ae437 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f9 expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.522007:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:2.0:1788845040.522090:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641717376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2866us (2986us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.522099:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 980 00000100:00100000:3.0:1788845040.525473:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.525612:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641717504 00000100:00100000:2.0:1788845040.525622:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641717504 00000100:00100000:2.0:1788845040.525678:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 981 00000100:00100000:2.0:1788845040.525687:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641717504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.525706:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.525732:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae43e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a00 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.525791:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660353024:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.525865:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.528464:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660353024:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.528503:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae43e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a00 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.528527:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae43e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a00 expref: 11 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.528542:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845040.528704:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641717504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3016us (3239us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.528721:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 981 00000100:00100000:0.0:1788845040.532863:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.533060:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641717632 00000100:00100000:3.0:1788845040.533072:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641717632 00000100:00100000:3.0:1788845040.533172:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1392 00000100:00100000:3.0:1788845040.533182:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2368:x1875739641717632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.533203:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.533223:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae445 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a07 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.533280:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae445 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a07 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845040.533294:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:3.0:1788845040.533303:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae445 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a07 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.533318:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.533321:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845040.533325:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845040.533426:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641717632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 244us (572us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845040.533434:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1392 00000100:00100000:1.0:1788845040.537948:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.538049:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641717760 00000100:00100000:2.0:1788845040.538060:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641717760 00000100:00100000:2.0:1788845040.538118:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 982 00000100:00100000:2.0:1788845040.538128:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641717760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.538150:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.538167:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a0e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.538201:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.538211:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 14) 00010000:00010000:2.0:1788845040.538216:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 14) 00010000:00010000:2.0:1788845040.538221:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1458176->1470463] (req 1458176->1470463) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.538271:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.538290:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 7 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.538305:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660353152:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845040.538382:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.541990:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660353152:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845040.542031:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1458176->1470463] (req 1458176->1470463) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.542052:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:2.0:1788845040.542118:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641717760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3991us (4181us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.542128:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 982 00000100:00100000:0.0:1788845040.543098:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.543557:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641717888 00000100:00100000:2.0:1788845040.543566:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641717888 00010000:00010000:2.0:1788845040.543617:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 8 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.543662:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 481 00000100:00100000:2.0:1788845040.543671:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d11050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641717888:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845040.543684:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845040.543707:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 8 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.543736:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 8 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.543767:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000468fe2b6/0xb2187301287ae3ff lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99cf expref: 8 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.543786:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 14) 00010000:00010000:2.0:1788845040.543790:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 14) 00010000:00010000:2.0:1788845040.543799:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.543810:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.543846:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.543861:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660353280:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845040.543968:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.549966:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.550069:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660353280:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845040.550096:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845040.550164:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d11050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641717888:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6494us (7068us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.550178:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 481 00000100:00100000:1.0:1788845040.550868:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641718016 00000100:00100000:1.0:1788845040.550877:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641718016 00002000:00100000:1.0:1788845040.550952:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7df80 x1875739641718016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.550963:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7df80 with x1875739641718016 00000100:00100000:1.0:1788845040.550986:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 912 00000100:00100000:1.0:1788845040.550994:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641718016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.551424:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641718016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 428us (1467us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.551433:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 912 00002000:00100000:1.0:1788845040.551440:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035b7df80 x1875739641718016/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.551447:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035b7df80 with x1875739641718016 00000100:00100000:0.0:1788845040.560174:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.560389:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641718144 00000100:00100000:3.0:1788845040.560400:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641718144 00000100:00100000:3.0:1788845040.560521:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 983 00000100:00100000:3.0:1788845040.560534:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641718144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.560556:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.560576:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae453 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a15 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.560650:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660353408:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845040.560755:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22000 going to sleep for 11 seconds 00002000:00010000:3.0:1788845040.563587:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1469132 00002000:00010000:3.0:1788845040.563597:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845039 -> 1788845040 00000100:00100000:3.0:1788845040.563622:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660353408:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845040.563659:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae453 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a15 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.563685:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae453 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a15 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.563701:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:3.0:1788845040.563844:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641718144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3312us (3679us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845040.563853:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 983 00000100:00100000:1.0:1788845040.567055:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.567139:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641718272 00000100:00100000:3.0:1788845040.567148:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641718272 00000100:00100000:3.0:1788845040.567208:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 984 00000100:00100000:3.0:1788845040.567219:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641718272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.567248:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.567266:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae45a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a1c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.567326:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660353536:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845040.567416:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22000 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.569523:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660353536:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845040.569560:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae45a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a1c expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.569585:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae45a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a1c expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.569612:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:3.0:1788845040.569672:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641718272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2453us (2625us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845040.569681:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 984 00000100:00100000:0.0:1788845040.583019:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.583066:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.583145:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641718528 00000100:00100000:3.0:1788845040.583155:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641718528 00002000:00100000:3.0:1788845040.583208:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be5f80 x1875739641718528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 520/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.583218:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be5f80 with x1875739641718528 00000100:00100000:3.0:1788845040.583278:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 913 00000100:00100000:3.0:1788845040.583289:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641718528:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.586210:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641718400 00000100:00100000:2.0:1788845040.586225:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641718400 00002000:00100000:2.0:1788845040.586299:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1458175) req@ffff94a035b7ea00 x1875739641718400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.586310:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7ea00 with x1875739641718400 ext(1048576->1458175) 00002000:00010000:2.0:1788845040.586315:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7ea00 00000100:00100000:2.0:1788845040.586351:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 469 00000100:00100000:2.0:1788845040.586362:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641718400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845040.590664:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641718528:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 7367us (7586us total) trans 0 rc 368640/368640 00000100:00100000:3.0:1788845040.590677:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 913 00002000:00100000:3.0:1788845040.590690:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be5f80 x1875739641718528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 520/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 368640/368640 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.590702:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be5f80 with x1875739641718528 00000100:00100000:2.0:1788845040.591601:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641718400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 5238us (8587us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.591611:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 469 00002000:00100000:2.0:1788845040.591620:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1458175) req@ffff94a035b7ea00 x1875739641718400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.591629:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035b7ea00 with x1875739641718400 ext(1048576->1458175) 00002000:00010000:2.0:1788845040.591634:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035b7ea00 00000100:00100000:0.0:1788845040.597605:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.597829:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641718656 00000100:00100000:2.0:1788845040.597840:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641718656 00000100:00100000:2.0:1788845040.597928:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 985 00000100:00100000:2.0:1788845040.597939:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641718656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.597960:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.597980:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae461 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a23 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.598039:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660353664:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.598127:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.605979:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660353664:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.606017:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae461 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a23 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.606041:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae461 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a23 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.606055:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:2.0:1788845040.606105:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641718656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8168us (8518us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.606113:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 985 00000100:00100000:1.0:1788845040.609480:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.609657:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641718784 00000100:00100000:3.0:1788845040.609666:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641718784 00000100:00100000:3.0:1788845040.609764:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 986 00000100:00100000:3.0:1788845040.609776:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641718784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.609796:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.609817:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae468 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a2a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.609879:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660353792:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.611319:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.614469:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660353792:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.614518:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae468 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a2a expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.614546:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae468 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a2a expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.614562:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:2.0:1788845040.614637:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641718784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4863us (5165us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.614646:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 986 00000100:00100000:1.0:1788845040.619580:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.619828:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641718912 00000100:00100000:3.0:1788845040.619838:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641718912 00000100:00100000:2.0:1788845040.620004:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 987 00000100:00100000:2.0:1788845040.620017:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641718912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.620038:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.620057:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a31 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.620092:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.620102:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 14) 00010000:00010000:2.0:1788845040.620107:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 14) 00010000:00010000:2.0:1788845040.620113:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1208320->1245183] (req 1208320->1245183) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.620149:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.620168:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 10 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.620195:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660353920:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845040.620296:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.622474:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660353920:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845040.622512:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1208320->1245183] (req 1208320->1245183) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.622533:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:2.0:1788845040.622603:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641718912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2579us (3028us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.622612:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 987 00000100:00100000:0.0:1788845040.623165:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.623810:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641719040 00000100:00100000:2.0:1788845040.623821:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641719040 00002000:00100000:2.0:1788845040.623870:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1458176->1469131) req@ffff94a035be7800 x1875739641719040/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 11448/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.623917:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be7800 with x1875739641719040 ext(1458176->1469131) 00010000:00010000:2.0:1788845040.623928:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 11 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.623958:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 11 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845040.623966:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a035be7800 00000100:00100000:2.0:1788845040.623996:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 27 00000100:00100000:2.0:1788845040.624006:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641719040:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845040.625063:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641719040:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 1052us (1894us total) trans 4294967470 rc 0/0 00000100:00100000:2.0:1788845040.625073:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 27 00002000:00100000:2.0:1788845040.625082:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1458176->1469131) req@ffff94a035be7800 x1875739641719040/t4294967470(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 11448/448 e 0 to 0 dl 1788845051 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.625091:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be7800 with x1875739641719040 ext(1458176->1469131) 00010000:00010000:2.0:1788845040.625099:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 13 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.625116:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 13 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845040.625129:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a035be7800 00000100:00100000:0.0:1788845040.633450:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.633632:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641719168 00000100:00100000:1.0:1788845040.633642:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641719168 00010000:00010000:1.0:1788845040.633684:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 13 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.633736:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 482 00000100:00100000:1.0:1788845040.633746:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641719168:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845040.633763:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845040.633785:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 13 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.633804:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 13 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.633827:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287ae44c lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1458176->1470463) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a0e expref: 13 pid: 6685 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.633843:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 14) 00010000:00010000:1.0:1788845040.633848:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 14) 00010000:00010000:1.0:1788845040.633856:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.633865:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.634095:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.634111:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660354048:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845040.634195:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.642508:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.642549:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.642853:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641719296 00000100:00100000:2.0:1788845040.642863:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641719296 00002000:00100000:2.0:1788845040.642933:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1470463) req@ffff94a035be4a80 x1875739641719296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.642945:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be4a80 with x1875739641719296 ext(1048576->1470463) 00002000:00010000:2.0:1788845040.642951:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be4a80 00000100:00100000:2.0:1788845040.642985:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 470 00000100:00100000:2.0:1788845040.642996:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641719296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845040.647108:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641719424 00000100:00100000:3.0:1788845040.647123:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641719424 00002000:00100000:3.0:1788845040.647196:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be4700 x1875739641719424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.647208:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be4700 with x1875739641719424 00000100:00100000:3.0:1788845040.647451:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 914 00000100:00100000:3.0:1788845040.647462:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641719424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.648983:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660354048:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845040.649025:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845040.649060:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641719168:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 15315us (15620us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.649071:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 482 00000100:00100000:2.0:1788845040.655242:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641719296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12225us (12721us total) trans 0 rc 421888/421888 00000100:00100000:2.0:1788845040.655264:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 470 00002000:00100000:2.0:1788845040.655279:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1470463) req@ffff94a035be4a80 x1875739641719296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 421888/421888 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.655290:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be4a80 with x1875739641719296 ext(1048576->1470463) 00002000:00010000:2.0:1788845040.655297:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be4a80 00000100:00100000:2.0:1788845040.657638:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641719424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 10168us (15077us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.657651:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 914 00002000:00100000:2.0:1788845040.657662:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be4700 x1875739641719424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.657671:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be4700 with x1875739641719424 00000100:00100000:0.0:1788845040.668372:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.668502:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641719552 00000100:00100000:2.0:1788845040.668514:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641719552 00000100:00100000:2.0:1788845040.668573:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 988 00000100:00100000:2.0:1788845040.668583:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641719552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.668674:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.668693:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae476 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a38 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.668765:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660354176:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.669166:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.678263:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660354176:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.678308:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae476 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a38 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.678336:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae476 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a38 expref: 13 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.678352:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845040.678473:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641719552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9890us (10300us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.678482:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 988 00000100:00100000:2.0:1788845040.686623:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.686745:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641719680 00000100:00100000:0.0:1788845040.686756:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641719680 00000100:00100000:0.0:1788845040.686818:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 989 00000100:00100000:0.0:1788845040.686828:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034508c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641719680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.686848:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.686867:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae47d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a3f expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.686967:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660354304:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.687045:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.690089:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660354304:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845040.690130:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae47d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a3f expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.690155:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae47d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a3f expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.690169:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845040.690226:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034508c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641719680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3399us (3613us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.690235:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 989 00000100:00100000:1.0:1788845040.696648:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.697255:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641719808 00000100:00100000:0.0:1788845040.697429:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641719808 00000100:00100000:0.0:1788845040.697497:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 990 00000100:00100000:0.0:1788845040.697508:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641719808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.697528:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.697547:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a46 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.697582:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.697602:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 14) 00010000:00010000:0.0:1788845040.697606:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 14) 00010000:00010000:0.0:1788845040.697612:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1134592->1200127] (req 1134592->1200127) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.697646:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.697665:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 7 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.697679:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660354432:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845040.697779:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.700864:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.701152:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641719936 00000100:00100000:3.0:1788845040.701161:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641719936 00002000:00100000:3.0:1788845040.701220:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1208320->1245183) req@ffff94a03f655f80 x1875739641719936/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.701236:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f655f80 with x1875739641719936 ext(1208320->1245183) 00010000:00010000:3.0:1788845040.701247:0:13321:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 8 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.701279:0:13321:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 8 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845040.701288:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f655f80 00000100:00100000:3.0:1788845040.701318:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 28 00000100:00100000:3.0:1788845040.701327:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641719936:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845040.702639:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660354432:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845040.702674:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1134592->1200127] (req 1134592->1200127) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.702694:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845040.702762:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641719808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5255us (6124us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.702770:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 990 00000100:00100000:3.0:1788845040.714073:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641719936:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 12734us (13207us total) trans 4294967471 rc 0/0 00000100:00100000:3.0:1788845040.714090:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 28 00002000:00100000:3.0:1788845040.714106:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1208320->1245183) req@ffff94a03f655f80 x1875739641719936/t4294967471(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/448 e 0 to 0 dl 1788845051 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.714117:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f655f80 with x1875739641719936 ext(1208320->1245183) 00010000:00010000:3.0:1788845040.714130:0:13321:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 10 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.714155:0:13321:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 10 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845040.714163:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f655f80 00000100:00100000:1.0:1788845040.721008:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.721117:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641720064 00000100:00100000:0.0:1788845040.721128:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641720064 00010000:00010000:0.0:1788845040.721169:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 10 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.721222:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 483 00000100:00100000:0.0:1788845040.721232:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641720064:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845040.721248:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845040.721283:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 10 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.721303:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 10 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.721333:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae46f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1208320->1245183) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a31 expref: 10 pid: 6684 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.721351:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 14) 00010000:00010000:0.0:1788845040.721356:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 14) 00010000:00010000:0.0:1788845040.721374:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 12 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.721384:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 12 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.721423:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 12 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.721438:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660354560:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845040.721519:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.727165:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.727391:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660354560:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845040.727417:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845040.727475:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641720064:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6243us (6476us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.727486:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 483 00000100:00100000:2.0:1788845040.727567:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641720192 00000100:00100000:2.0:1788845040.727574:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641720192 00000100:00100000:2.0:1788845040.727645:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:2.0:1788845040.727656:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641720192:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:2.0:1788845040.727997:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1469132 -> 1469132 00002000:00010000:2.0:1788845040.728002:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00002000:00010000:2.0:1788845040.728004:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845040 -> 1788845040 00002000:00010000:2.0:1788845040.728006:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00002000:00010000:2.0:1788845040.728008:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:2.0:1788845040.728037:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641720192:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 382us (880us total) trans 4294967472 rc 0/0 00000100:00100000:2.0:1788845040.728045:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 471 00000100:00100000:1.0:1788845040.733576:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.734355:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641720320 00000100:00100000:2.0:1788845040.734376:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641720320 00000100:00100000:2.0:1788845040.734495:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 991 00000100:00100000:2.0:1788845040.734506:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641720320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.734523:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.734541:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae48b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a4d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.734597:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660354688:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.734678:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.741774:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660354688:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.741818:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae48b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a4d expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.741845:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae48b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a4d expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.741864:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845040.742216:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641720320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7711us (8649us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.742226:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 991 00000100:00100000:1.0:1788845040.747827:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.747982:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641720448 00000100:00100000:2.0:1788845040.747993:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641720448 00000100:00100000:2.0:1788845040.748091:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 992 00000100:00100000:2.0:1788845040.748103:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641720448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.748122:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.748141:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae492 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a54 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.748212:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660354816:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.748301:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.758789:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660354816:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.758834:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae492 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a54 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.758860:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae492 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a54 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.758875:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:2.0:1788845040.758976:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641720448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10874us (11156us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.758986:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 992 00000100:00100000:1.0:1788845040.763645:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.763842:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641720576 00000100:00100000:0.0:1788845040.763852:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641720576 00000100:00100000:0.0:1788845040.763925:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 993 00000100:00100000:0.0:1788845040.763935:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641720576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.763952:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.763969:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287ae499 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a5b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.764022:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660354944:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.764088:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.766660:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660354944:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845040.766701:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287ae499 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a5b expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.766738:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287ae499 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a5b expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.766752:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845040.766812:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641720576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2877us (3174us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.766820:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 993 00000100:00100000:1.0:1788845040.771996:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.772107:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641720704 00000100:00100000:0.0:1788845040.772118:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641720704 00002000:00100000:0.0:1788845040.772168:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035742a00 x1875739641720704/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.772179:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742a00 with x1875739641720704 00000100:00100000:0.0:1788845040.772214:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 915 00000100:00100000:0.0:1788845040.772224:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641720704:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.772457:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641720704:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (470us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845040.772464:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 915 00002000:00100000:0.0:1788845040.772471:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035742a00 x1875739641720704/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.772478:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035742a00 with x1875739641720704 00000100:00100000:1.0:1788845040.779416:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.779525:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641720832 00000100:00100000:2.0:1788845040.779537:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641720832 00002000:00100000:2.0:1788845040.779590:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413b80 x1875739641720832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.779602:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413b80 with x1875739641720832 00000100:00100000:2.0:1788845040.779640:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 916 00000100:00100000:2.0:1788845040.779649:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641720832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.779946:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641720832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 296us (540us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845040.779953:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 916 00002000:00100000:2.0:1788845040.779960:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413b80 x1875739641720832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.779969:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413b80 with x1875739641720832 00000100:00100000:1.0:1788845040.784215:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.784320:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641720960 00000100:00100000:0.0:1788845040.784332:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641720960 00002000:00100000:0.0:1788845040.784384:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413480 x1875739641720960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.784395:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413480 with x1875739641720960 00000100:00100000:0.0:1788845040.784432:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 917 00000100:00100000:0.0:1788845040.784442:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641720960:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.784693:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641720960:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 251us (487us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845040.784700:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 917 00002000:00100000:0.0:1788845040.784707:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03413480 x1875739641720960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.784725:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03413480 with x1875739641720960 00000100:00100000:3.0:1788845040.787694:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.788359:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641721088 00000100:00100000:2.0:1788845040.788371:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641721088 00002000:00100000:2.0:1788845040.788424:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0e00 x1875739641721088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.788435:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0e00 with x1875739641721088 00000100:00100000:2.0:1788845040.788523:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 918 00000100:00100000:2.0:1788845040.788533:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641721088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.788795:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641721088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 263us (1109us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845040.788803:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 918 00002000:00100000:2.0:1788845040.788811:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0e00 x1875739641721088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.788819:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0e00 with x1875739641721088 00000100:00100000:1.0:1788845040.792606:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.792722:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641721216 00000100:00100000:0.0:1788845040.792734:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641721216 00002000:00100000:0.0:1788845040.792786:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03412680 x1875739641721216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.792798:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03412680 with x1875739641721216 00000100:00100000:0.0:1788845040.792841:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 919 00000100:00100000:0.0:1788845040.792852:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641721216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.793105:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641721216:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (517us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845040.793113:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 919 00002000:00100000:0.0:1788845040.793119:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03412680 x1875739641721216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.793128:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03412680 with x1875739641721216 00000100:00100000:0.0:1788845040.799144:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.799272:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641721344 00000100:00100000:1.0:1788845040.799283:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641721344 00002000:00100000:1.0:1788845040.799335:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be6a00 x1875739641721344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.799347:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be6a00 with x1875739641721344 00000100:00100000:1.0:1788845040.799386:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 920 00000100:00100000:1.0:1788845040.799397:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641721344:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.799627:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641721344:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (491us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845040.799635:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 920 00002000:00100000:1.0:1788845040.799642:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035be6a00 x1875739641721344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.799649:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035be6a00 with x1875739641721344 00000100:00100000:1.0:1788845040.804682:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.804781:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641721472 00000100:00100000:3.0:1788845040.804792:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641721472 00002000:00100000:3.0:1788845040.804846:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410700 x1875739641721472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.804858:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410700 with x1875739641721472 00000100:00100000:3.0:1788845040.804934:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 921 00000100:00100000:3.0:1788845040.804945:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641721472:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845040.805160:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641721472:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 216us (488us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845040.805168:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 921 00002000:00100000:3.0:1788845040.805176:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410700 x1875739641721472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.805183:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410700 with x1875739641721472 00000100:00100000:2.0:1788845040.809133:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.809223:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641721600 00000100:00100000:1.0:1788845040.809234:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641721600 00002000:00100000:1.0:1788845040.809302:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f655c00 x1875739641721600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.809315:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f655c00 with x1875739641721600 00000100:00100000:1.0:1788845040.809359:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 922 00000100:00100000:1.0:1788845040.809369:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641721600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845040.809756:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641721600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 388us (633us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845040.809764:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 922 00002000:00100000:1.0:1788845040.809772:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f655c00 x1875739641721600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.809780:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f655c00 with x1875739641721600 00000100:00100000:0.0:1788845040.815465:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.815608:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641721728 00000100:00100000:2.0:1788845040.815617:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641721728 00000100:00100000:2.0:1788845040.815671:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 994 00000100:00100000:2.0:1788845040.815681:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641721728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.815698:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.815724:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4a0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a62 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.815778:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660355072:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.815855:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.817704:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660355072:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.817750:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4a0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a62 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.817775:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4a0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a62 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.817793:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845040.817851:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641721728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2171us (2396us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.817859:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 994 00000100:00100000:3.0:1788845040.820148:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.820531:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641721856 00000100:00100000:2.0:1788845040.820542:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641721856 00000100:00100000:2.0:1788845040.820609:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 995 00000100:00100000:2.0:1788845040.820619:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641721856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.820636:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.820654:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae4a7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a69 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.820727:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660355200:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.820966:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.824559:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660355200:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.824611:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae4a7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a69 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.824637:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae4a7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a69 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.824650:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:2.0:1788845040.824723:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641721856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4105us (4584us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.824733:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 995 00000100:00100000:3.0:1788845040.828618:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845040.828702:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641721984 00000100:00100000:1.0:1788845040.828738:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641721984 00000100:00100000:1.0:1788845040.828797:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 996 00000100:00100000:1.0:1788845040.828808:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641721984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845040.828830:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845040.828849:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae4ae lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a70 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845040.828936:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660355328:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845040.829021:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845040.831572:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660355328:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845040.831627:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae4ae lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a70 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.831653:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae4ae lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a70 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845040.831667:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845040.831734:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641721984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2927us (3125us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845040.831743:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 996 00000100:00100000:3.0:1788845040.835280:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.835362:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641722112 00000100:00100000:0.0:1788845040.835373:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641722112 00000100:00100000:0.0:1788845040.835434:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 997 00000100:00100000:0.0:1788845040.835444:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641722112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.835464:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.835482:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae4b5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a77 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.835540:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660355456:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.835635:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.838445:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660355456:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845040.838487:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae4b5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a77 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.838511:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae4b5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a77 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.838525:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006800, rc 0) 00000100:00100000:0.0:1788845040.838582:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641722112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3138us (3311us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.838601:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 997 00000100:00100000:1.0:1788845040.842521:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.842626:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641722240 00000100:00100000:2.0:1788845040.842637:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641722240 00000100:00100000:2.0:1788845040.842717:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 472 00000100:00100000:2.0:1788845040.842728:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641722240:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:2.0:1788845040.843255:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1469132 -> 1469132 00002000:00010000:2.0:1788845040.843259:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00002000:00010000:2.0:1788845040.843262:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845040 -> 1788845040 00002000:00010000:2.0:1788845040.843264:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00002000:00010000:2.0:1788845040.843266:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:2.0:1788845040.843335:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641722240:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 608us (822us total) trans 4294967473 rc 0/0 00000100:00100000:2.0:1788845040.843345:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 472 00000100:00100000:3.0:1788845040.847530:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.847641:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641722368 00000100:00100000:2.0:1788845040.847652:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641722368 00000100:00100000:2.0:1788845040.847718:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 998 00000100:00100000:2.0:1788845040.847731:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641722368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.847750:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.847767:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae4bc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a7e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.847826:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660355584:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.847949:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.849799:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660355584:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.849837:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae4bc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a7e expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.849862:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae4bc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a7e expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.849979:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:2.0:1788845040.850038:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641722368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2311us (2516us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.850046:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 998 00000100:00100000:1.0:1788845040.850225:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660356096:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850281:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845040.850310:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788845040.850315:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660355712:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850340:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845040.850363:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03412300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660356224:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850389:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845040.850412:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660355840:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850435:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845040.850452:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660356352:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850473:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845040.850489:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788845040.850494:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660355968:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850514:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845040.850543:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660355712 00000100:00100000:1.0:1788845040.850548:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660355712 00000100:00100000:1.0:1788845040.850600:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:1.0:1788845040.850609:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660355712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850651:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660355712:12345-0@lo:400:kworker.0 Request processed in 43us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.850656:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:1.0:1788845040.850692:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660355712:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850725:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660356224 00000100:00100000:1.0:1788845040.850728:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660356224 00000100:00080000:1.0:1788845040.850748:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845040.850776:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660355840 00000100:00100000:1.0:1788845040.850779:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660355840 00000100:00100000:1.0:1788845040.850794:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660355968 00000100:00100000:1.0:1788845040.850797:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660355968 00000100:00100000:1.0:1788845040.850816:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:1.0:1788845040.850821:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660356224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850858:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660356224:12345-0@lo:400:kworker.0 Request processed in 36us (467us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.850867:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:1.0:1788845040.850879:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:1.0:1788845040.850911:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660355840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850946:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660355840:12345-0@lo:400:kworker.0 Request processed in 36us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.850952:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:1.0:1788845040.850962:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:1.0:1788845040.850969:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660355968:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.850997:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660355968:12345-0@lo:400:kworker.0 Request processed in 29us (481us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845040.851002:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00080000:1.0:1788845040.851015:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845040.851034:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660355968:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.851043:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660355840:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.851068:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03412300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660356224:0@lo:400:kworker.0 00000100:00100000:2.0:1788845040.851300:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660356096 00000100:00100000:2.0:1788845040.851305:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660356096 00000100:00100000:2.0:1788845040.851331:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660356352 00000100:00100000:2.0:1788845040.851335:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660356352 00000100:00100000:2.0:1788845040.851355:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00100000:2.0:1788845040.851362:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660356096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845040.851400:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660356096:12345-0@lo:400:kworker.0 Request processed in 39us (1119us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.851406:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00000100:00100000:2.0:1788845040.851420:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:2.0:1788845040.851426:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660356352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845040.851456:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660356352:12345-0@lo:400:kworker.0 Request processed in 30us (982us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.851462:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00080000:1.0:1788845040.851664:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788845040.851667:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845040.851687:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660356352:0@lo:400:kworker.0 00000100:00100000:1.0:1788845040.851722:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03411880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660356096:0@lo:400:kworker.0 00000100:00100000:0.0:1788845040.854805:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.854900:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641722496 00000100:00100000:2.0:1788845040.854910:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641722496 00000100:00100000:2.0:1788845040.854968:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 999 00000100:00100000:2.0:1788845040.854978:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641722496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.855000:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.855021:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4c3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a85 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.855089:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660356480:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.855177:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.857090:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660356480:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.857128:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4c3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a85 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.857150:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4c3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a85 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.857161:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845040.857209:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641722496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2233us (2415us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.857217:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 999 00000100:00100000:1.0:1788845040.860506:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.860769:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641722624 00000100:00100000:2.0:1788845040.860779:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641722624 00000100:00100000:2.0:1788845040.860830:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1000 00000100:00100000:2.0:1788845040.860841:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641722624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.860861:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.860905:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000417030e2/0xb2187301287ae4ca lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a8c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.860968:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660356608:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.861058:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.863356:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660356608:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845040.863401:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000417030e2/0xb2187301287ae4ca lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a8c expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.863426:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000417030e2/0xb2187301287ae4ca lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a8c expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.863439:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032200, rc 0) 00000100:00100000:2.0:1788845040.863532:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641722624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2692us (3035us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.863541:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1000 00000100:00100000:1.0:1788845040.872044:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.872120:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641722752 00000100:00100000:3.0:1788845040.872130:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641722752 00000100:00100000:3.0:1788845040.872223:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1393 00000100:00100000:3.0:1788845040.872245:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641722752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845040.872278:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845040.872282:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845040.872337:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae445 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a07 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.872359:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.872364:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845040.872379:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a93 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.872405:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.872415:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.872455:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.872472:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 14 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845040.872486:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660356736:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845040.872577:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.877098:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.877328:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660356736:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845040.877368:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.877390:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.877393:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845040.877398:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.877408:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845040.877478:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641722752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5236us (5443us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845040.877487:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1393 00000100:00100000:2.0:1788845040.877784:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641722880 00000100:00100000:2.0:1788845040.877791:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641722880 00010000:00010000:2.0:1788845040.877827:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 15 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.877864:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 484 00000100:00100000:2.0:1788845040.877873:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641722880:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845040.877905:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845040.877933:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 15 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.877949:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 15 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.877970:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae422 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99eb expref: 15 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.877984:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.877989:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.877999:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.878008:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.878041:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.878054:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660356864:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845040.878106:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.881489:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660356864:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845040.881515:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845040.881569:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641722880:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3696us (4480us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.881594:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 484 00000100:00100000:3.0:1788845040.884272:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.884603:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641723008 00000100:00100000:0.0:1788845040.884613:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641723008 00002000:00100000:0.0:1788845040.884664:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb3100 x1875739641723008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.884677:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb3100 with x1875739641723008 00000100:00100000:0.0:1788845040.884726:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 923 00000100:00100000:0.0:1788845040.884737:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641723008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.885196:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641723008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 459us (931us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845040.885204:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 923 00002000:00100000:0.0:1788845040.885211:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb3100 x1875739641723008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.885219:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb3100 with x1875739641723008 00000100:00100000:1.0:1788845040.895121:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.895411:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641723136 00000100:00100000:0.0:1788845040.895421:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641723136 00002000:00100000:0.0:1788845040.895471:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410000 x1875739641723136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.895483:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410000 with x1875739641723136 00000100:00100000:0.0:1788845040.895523:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 924 00000100:00100000:0.0:1788845040.895534:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641723136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.895834:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641723136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 301us (725us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845040.895843:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 924 00002000:00100000:0.0:1788845040.895850:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410000 x1875739641723136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.895857:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410000 with x1875739641723136 00000100:00100000:3.0:1788845040.899698:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.899801:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641723264 00000100:00100000:2.0:1788845040.899811:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641723264 00002000:00100000:2.0:1788845040.899863:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0a80 x1875739641723264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.899874:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0a80 with x1875739641723264 00000100:00100000:2.0:1788845040.899941:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 925 00000100:00100000:2.0:1788845040.899951:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641723264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.900218:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2370:x1875739641723264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (529us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845040.900226:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 925 00002000:00100000:2.0:1788845040.900243:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0a80 x1875739641723264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.900251:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0a80 with x1875739641723264 00000100:00100000:3.0:1788845040.903450:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.903691:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641723392 00000100:00100000:0.0:1788845040.903703:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641723392 00002000:00100000:0.0:1788845040.903764:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb3480 x1875739641723392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.903783:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb3480 with x1875739641723392 00000100:00100000:0.0:1788845040.903827:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 926 00000100:00100000:0.0:1788845040.903839:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641723392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.904122:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641723392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 285us (681us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845040.904131:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 926 00002000:00100000:0.0:1788845040.904138:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb3480 x1875739641723392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 488/4536 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.904145:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb3480 with x1875739641723392 00000100:00100000:3.0:1788845040.909022:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.909222:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641723520 00000100:00100000:2.0:1788845040.909233:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641723520 00002000:00100000:2.0:1788845040.909291:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb2d80 x1875739641723520/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 16872/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.909304:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb2d80 with x1875739641723520 00000100:00100000:2.0:1788845040.909349:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 927 00000100:00100000:2.0:1788845040.909359:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641723520:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845040.909792:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 411260 -> 411260 00000004:00010000:2.0:1788845040.909797:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:2.0:1788845040.909799:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845039 -> 1788845039 00000004:00010000:2.0:1788845040.909801:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:2.0:1788845040.909803:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 648 00000100:00100000:2.0:1788845040.909868:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641723520:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 509us (855us total) trans 4294967799 rc 0/0 00000100:00100000:2.0:1788845040.909876:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 927 00002000:00100000:2.0:1788845040.909924:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb2d80 x1875739641723520/t4294967799(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:146/0 lens 16872/448 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.909933:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb2d80 with x1875739641723520 00000100:00100000:3.0:1788845040.912831:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.912986:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641723648 00000100:00100000:2.0:1788845040.912997:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641723648 00000100:00100000:2.0:1788845040.913064:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1394 00000100:00100000:2.0:1788845040.913075:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2369:x1875739641723648:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845040.944953:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967792 is committed 00000001:00080000:1.0:1788845040.944976:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967797 is committed 00000001:00080000:1.0:1788845040.945004:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967799 is committed 00000100:00100000:2.0:1788845040.945309:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641723648:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 32230us (32482us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.945321:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1394 00000100:00100000:1.0:1788845040.948523:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.948700:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641723776 00000100:00100000:2.0:1788845040.948718:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641723776 00000100:00100000:2.0:1788845040.948821:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1395 00000100:00100000:2.0:1788845040.948833:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641723776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845040.948853:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845040.948873:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4d8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9a9a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.948991:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.949036:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660356992:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845040.949159:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000004:00010000:2.0:1788845040.951442:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845039 -> 1788845040 00000100:00100000:2.0:1788845040.951475:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660356992:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845040.951499:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:2.0:1788845040.951517:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4d8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a9a expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.951540:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.951544:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845040.951549:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a9a expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.951564:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032200, rc 0) 00000100:00100000:2.0:1788845040.951762:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641723776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2930us (3249us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845040.951770:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1395 00000100:00100000:1.0:1788845040.954562:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.954725:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641723904 00000100:00100000:0.0:1788845040.954735:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641723904 00000100:00100000:0.0:1788845040.954830:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1396 00000100:00100000:0.0:1788845040.954841:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641723904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.954865:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.954903:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae4df lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9aa1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.954940:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.954983:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660357120:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845040.955063:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845040.957188:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660357120:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845040.957216:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:0.0:1788845040.957392:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae4df lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa1 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.957415:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845040.957418:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845040.957423:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae4df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa1 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.957437:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845040.957532:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641723904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2691us (2977us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845040.957541:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1396 00000100:00100000:1.0:1788845040.962726:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845040.962819:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641724032 00000100:00100000:0.0:1788845040.962829:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641724032 00000100:00100000:0.0:1788845040.962958:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1397 00000100:00100000:0.0:1788845040.962970:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641724032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845040.962992:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845040.963014:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9aa8 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.963047:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.963056:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa8 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.963088:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845040.963110:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 14 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845040.963123:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660357248:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845040.963188:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845040.966175:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660357248:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845040.966212:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa8 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.966229:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845040.966238:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845040.966243:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa8 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845040.966260:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:3.0:1788845040.966320:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641724032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3353us (3613us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845040.966330:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1397 00000100:00100000:1.0:1788845040.967832:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.968021:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641724160 00000100:00100000:2.0:1788845040.968030:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641724160 00010000:00010000:2.0:1788845040.968068:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 15 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.968134:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 485 00000100:00100000:2.0:1788845040.968144:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641724160:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845040.968157:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845040.968184:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 15 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.968201:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 15 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.968222:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae4d1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a93 expref: 15 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.968364:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845040.968372:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa8 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.968383:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa8 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.968392:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa8 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845040.968429:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa8 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845040.968442:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660357376:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845040.968520:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845040.975574:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660357376:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845040.975611:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845040.975670:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641724160:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7527us (7843us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845040.975681:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 485 00000100:00100000:2.0:1788845040.978921:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845040.978972:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845040.979065:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641724416 00000100:00100000:3.0:1788845040.979076:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641724416 00002000:00100000:3.0:1788845040.979131:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1470463) req@ffff94a03f656a00 x1875739641724416/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845040.979143:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f656a00 with x1875739641724416 ext(1048576->1470463) 00002000:00010000:3.0:1788845040.979150:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f656a00 00000100:00100000:3.0:1788845040.979182:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 473 00000100:00100000:3.0:1788845040.979192:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641724416:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845040.980414:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641724288 00000100:00100000:0.0:1788845040.980424:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641724288 00002000:00100000:0.0:1788845040.980465:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657480 x1875739641724288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/0 e 0 to 0 dl 1788845051 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845040.980476:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657480 with x1875739641724288 00000100:00100000:0.0:1788845040.980519:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 928 00000100:00100000:0.0:1788845040.980528:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641724288:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845040.994650:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641724416:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 15449us (15667us total) trans 0 rc 421888/421888 00000100:00100000:2.0:1788845040.994664:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 473 00002000:00100000:2.0:1788845040.994678:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1470463) req@ffff94a03f656a00 x1875739641724416/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 421888/421888 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845040.994689:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f656a00 with x1875739641724416 ext(1048576->1470463) 00002000:00010000:2.0:1788845040.994695:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f656a00 00000100:00100000:1.0:1788845040.997957:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641724288:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17419us (19035us total) trans 0 rc 413696/413696 00000100:00100000:1.0:1788845040.997970:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 928 00002000:00100000:1.0:1788845040.997984:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f657480 x1875739641724288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:146/0 lens 488/440 e 0 to 0 dl 1788845051 ref 1 fl Complete:/600/0 rc 413696/413696 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845040.997994:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f657480 with x1875739641724288 00000100:00100000:1.0:1788845041.000657:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.000769:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.000820:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641724672 00000100:00100000:2.0:1788845041.000829:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641724544 00000100:00100000:3.0:1788845041.000830:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641724672 00000100:00100000:2.0:1788845041.000833:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641724544 00000100:00100000:3.0:1788845041.000940:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1001 00000100:00100000:2.0:1788845041.000940:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1398 00000100:00100000:3.0:1788845041.000951:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641724672:12345-192.168.201.29@tcp:101:fsx.0 00000100:00100000:2.0:1788845041.000951:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641724544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.000968:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.000968:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.000985:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae4f4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ab6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.000985:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9aaf expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.001017:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845041.001028:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:2.0:1788845041.001034:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.001041:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660357504:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.001046:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.001049:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.001053:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032200, rc 0) 00000100:00100000:2.0:1788845041.001094:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641724544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 146us (449us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.001102:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1398 00000100:00080000:1.0:1788845041.001370:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788845041.001374:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845041.001398:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22600 going to sleep for 11 seconds 00002000:00010000:3.0:1788845041.003526:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845040 -> 1788845041 00000100:00100000:3.0:1788845041.003556:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660357504:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845041.003654:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae4f4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ab6 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.003684:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287ae4f4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ab6 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.003698:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845041.003802:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641724672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2852us (3031us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.003810:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1001 00000100:00100000:0.0:1788845041.006460:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.006569:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641724800 00000100:00100000:2.0:1788845041.006578:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641724800 00000100:00100000:2.0:1788845041.006647:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1002 00000100:00100000:2.0:1788845041.006657:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641724800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.006676:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.006694:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4fb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9abd expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.006759:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660357632:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.006972:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.009854:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660357632:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.010052:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4fb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9abd expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.010078:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae4fb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9abd expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.010091:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845041.010189:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641724800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3533us (3736us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.010197:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1002 00000100:00100000:0.0:1788845041.016201:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.016291:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641724928 00000100:00100000:2.0:1788845041.016301:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641724928 00000100:00100000:2.0:1788845041.016374:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1399 00000100:00100000:2.0:1788845041.016385:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641724928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.016405:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.016408:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845041.016449:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae4e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aa8 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.016466:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.016471:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845041.016486:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ac4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.016512:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.016522:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.016553:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.016570:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 14 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1788845041.016680:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845041.016765:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660357760:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845041.016848:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.020061:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.020146:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641725056 00000100:00100000:1.0:1788845041.020156:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641725056 00010000:00010000:1.0:1788845041.020195:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 15 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.020244:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:1.0:1788845041.020265:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641725056:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845041.020280:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845041.020306:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 15 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.020330:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 15 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.020352:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.020359:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.020371:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.020380:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.020419:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.020434:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5df800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660357888:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845041.020519:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788845041.021154:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae4ed lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aaf expref: 15 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.021173:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660357760:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845041.021196:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.021208:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.021211:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.021215:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:2.0:1788845041.021349:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641724928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4871us (5061us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.021359:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1399 00000100:00100000:2.0:1788845041.024013:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5df800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660357888:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845041.024045:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845041.024095:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641725056:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3841us (4043us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.024108:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:0.0:1788845041.026747:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.026830:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641725184 00000100:00100000:2.0:1788845041.026840:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641725184 00000100:00100000:2.0:1788845041.026926:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 929 00000100:00100000:2.0:1788845041.026936:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641725184:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845041.027170:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 411260 -> 411260 00000004:00010000:2.0:1788845041.027174:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845040 00000004:00010000:2.0:1788845041.027177:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845040 -> 1788845040 00000004:00010000:2.0:1788845041.027179:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845040 -> 1788845040 00000004:00010000:2.0:1788845041.027182:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 648 -> 518 00000100:00100000:2.0:1788845041.027258:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641725184:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 323us (520us total) trans 4294967800 rc 0/0 00000100:00100000:2.0:1788845041.027266:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 929 00000100:00100000:0.0:1788845041.031442:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.031492:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.031602:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641725312 00000100:00100000:2.0:1788845041.031612:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641725312 00000100:00100000:2.0:1788845041.031670:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1400 00000100:00100000:2.0:1788845041.031681:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641725312:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.031703:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.031732:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae509 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9acb expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.031774:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.031817:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660358016:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.031923:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.032089:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641725440 00000100:00100000:1.0:1788845041.032093:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641725440 00000100:00100000:1.0:1788845041.032137:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1003 00000100:00100000:1.0:1788845041.032145:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641725440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.032153:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.032165:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ae510 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ad2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.032208:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660358144:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.032239:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845041.035784:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845040 -> 1788845041 00000004:00010000:2.0:1788845041.035793:0:5839:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845040 -> 1788845041 00000100:00100000:2.0:1788845041.035818:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660358016:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845041.035842:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:2.0:1788845041.035862:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae509 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9acb expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.035916:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.035920:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.035925:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032200/0xb2187301287ae509 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9acb expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.035940:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032200, rc 0) 00000100:00100000:2.0:1788845041.036002:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641725312:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4322us (4569us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.036015:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1400 00002000:00010000:0.0:1788845041.036804:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845040 -> 1788845041 00000100:00100000:0.0:1788845041.036827:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660358144:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.036853:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ae510 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ad2 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.036872:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ae510 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ad2 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.036910:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:0.0:1788845041.037009:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641725440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4863us (5515us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.037017:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1003 00000100:00100000:1.0:1788845041.041322:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.041460:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641725568 00000100:00100000:3.0:1788845041.041470:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641725568 00000100:00100000:3.0:1788845041.041530:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1401 00000100:00100000:3.0:1788845041.041540:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641725568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.041562:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.041582:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae517 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ad9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.041628:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.041668:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660358272:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.041757:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.043642:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.043743:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641725696 00000100:00100000:1.0:1788845041.043753:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641725696 00000100:00100000:1.0:1788845041.043808:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1004 00000100:00100000:1.0:1788845041.043818:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641725696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.043836:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.043852:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae51e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ae0 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.043951:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660358400:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.044031:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.047025:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660358400:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.047066:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae51e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ae0 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.047091:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae51e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ae0 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.047105:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845041.047166:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641725696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3349us (3532us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.047178:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1004 00000100:00100000:0.0:1788845041.050119:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660358272:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845041.050150:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:0.0:1788845041.050170:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae517 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ad9 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.050191:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.050195:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845041.050200:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae517 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ad9 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.050215:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:0.0:1788845041.050294:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641725568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8753us (8980us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.050302:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1401 00000100:00100000:2.0:1788845041.054543:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.054693:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641725824 00000100:00100000:3.0:1788845041.054713:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641725824 00002000:00100000:3.0:1788845041.054764:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656d80 x1875739641725824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845041.054781:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656d80 with x1875739641725824 00000100:00100000:3.0:1788845041.054820:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 930 00000100:00100000:3.0:1788845041.054831:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641725824:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845041.055239:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641725824:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 408us (703us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845041.055247:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 930 00002000:00100000:3.0:1788845041.058754:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f656d80 x1875739641725824/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/4536 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845041.058776:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f656d80 with x1875739641725824 00000100:00100000:0.0:1788845041.059443:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.059539:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641725952 00000100:00100000:2.0:1788845041.059549:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641725952 00002000:00100000:2.0:1788845041.059659:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848a80 x1875739641725952/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.059669:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848a80 with x1875739641725952 00000100:00100000:2.0:1788845041.059737:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 931 00000100:00100000:2.0:1788845041.059748:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641725952:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845041.060029:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641725952:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (590us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845041.060037:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 931 00002000:00100000:2.0:1788845041.060044:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848a80 x1875739641725952/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/4536 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.060052:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848a80 with x1875739641725952 00000100:00100000:1.0:1788845041.064250:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.064379:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641726080 00000100:00100000:0.0:1788845041.064390:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641726080 00002000:00100000:0.0:1788845041.064444:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5df480 x1875739641726080/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.064455:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5df480 with x1875739641726080 00000100:00100000:0.0:1788845041.064499:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 932 00000100:00100000:0.0:1788845041.064511:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641726080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845041.064804:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641726080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (563us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845041.064813:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 932 00002000:00100000:0.0:1788845041.064820:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5df480 x1875739641726080/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/4536 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.064828:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5df480 with x1875739641726080 00000100:00100000:3.0:1788845041.068391:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.068603:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641726208 00000100:00100000:0.0:1788845041.068612:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641726208 00002000:00100000:0.0:1788845041.068657:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366680 x1875739641726208/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 12776/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.068668:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366680 with x1875739641726208 00000100:00100000:0.0:1788845041.068718:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 933 00000100:00100000:0.0:1788845041.068729:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641726208:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845041.069159:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 411260 -> 411260 00000004:00010000:0.0:1788845041.069163:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845040 -> 1788845041 00000004:00010000:0.0:1788845041.069166:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845040 00000004:00010000:0.0:1788845041.069168:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.069169:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:0.0:1788845041.069229:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641726208:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 501us (848us total) trans 4294967801 rc 0/0 00000100:00100000:0.0:1788845041.069237:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 933 00002000:00100000:0.0:1788845041.069245:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02366680 x1875739641726208/t4294967801(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 12776/448 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.069258:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02366680 with x1875739641726208 00000100:00100000:3.0:1788845041.071747:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.071925:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641726336 00000100:00100000:0.0:1788845041.071936:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641726336 00000100:00100000:0.0:1788845041.071996:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1402 00000100:00100000:0.0:1788845041.072007:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641726336:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:2.0:1788845041.104718:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967801 is committed 00000100:00100000:0.0:1788845041.104916:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641726336:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 32905us (33172us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.104930:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1402 00000100:00100000:1.0:1788845041.109696:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.109796:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.109912:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641726464 00000100:00100000:2.0:1788845041.109922:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641726464 00000100:00100000:2.0:1788845041.109981:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1403 00000100:00100000:2.0:1788845041.109992:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641726464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.110012:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.110034:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ae525 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ae7 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.110080:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.110120:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660358528:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.110356:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.110468:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641726592 00000100:00100000:3.0:1788845041.110474:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641726592 00000100:00100000:3.0:1788845041.110552:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1005 00000100:00100000:3.0:1788845041.110561:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641726592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.110571:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.110594:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae52c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9aee expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.110630:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660358656:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.110673:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788845041.113542:0:6684:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845040 -> 1788845041 00000100:00100000:3.0:1788845041.113574:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660358656:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845041.113619:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae52c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aee expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.113646:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae52c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9aee expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.113661:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:3.0:1788845041.113731:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641726592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3171us (3934us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.113739:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1005 00000004:00010000:1.0:1788845041.115444:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845040 -> 1788845041 00000004:00010000:1.0:1788845041.115455:0:11036:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 518 -> 584 00000100:00100000:1.0:1788845041.115481:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660358528:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845041.115503:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:1.0:1788845041.115521:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ae525 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ae7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.115540:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.115543:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845041.115547:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032800/0xb2187301287ae525 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ae7 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.115561:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032800, rc 0) 00000100:00100000:1.0:1788845041.115736:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641726464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5744us (6050us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.115744:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1403 00000100:00100000:3.0:1788845041.118669:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.118749:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.118845:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641726848 00000100:00100000:2.0:1788845041.118855:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641726848 00000100:00100000:2.0:1788845041.118934:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1006 00000100:00100000:2.0:1788845041.118944:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641726848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.118962:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.118980:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b0876318/0xb2187301287ae533 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9afc expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.119039:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660358784:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.119118:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.121496:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660358784:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845041.121533:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b0876318/0xb2187301287ae533 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9afc expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.121555:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b0876318/0xb2187301287ae533 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9afc expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.121569:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032600, rc 0) 00000100:00100000:3.0:1788845041.121664:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641726848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2720us (3091us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.121672:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1006 00000100:00100000:0.0:1788845041.121855:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641726720 00000100:00100000:0.0:1788845041.121862:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641726720 00000100:00100000:0.0:1788845041.121919:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1404 00000100:00100000:0.0:1788845041.121928:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641726720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.121944:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.121959:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae53a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9af5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.121990:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.122024:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660358912:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.122070:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.125198:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660358912:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845041.125405:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:3.0:1788845041.125426:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae53a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9af5 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.125447:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.125450:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.125455:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287ae53a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9af5 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.125469:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:3.0:1788845041.125574:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641726720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3647us (6824us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.125591:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1404 00000100:00100000:1.0:1788845041.129189:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.129334:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.129443:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641726976 00000100:00100000:3.0:1788845041.129454:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641726976 00000100:00100000:3.0:1788845041.129515:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1405 00000100:00100000:3.0:1788845041.129526:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641726976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.129548:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.129568:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae541 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b03 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.129616:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.129659:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660359040:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.129752:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.130085:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641727104 00000100:00100000:0.0:1788845041.130090:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641727104 00000100:00100000:0.0:1788845041.130160:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1007 00000100:00100000:0.0:1788845041.130170:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641727104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.130183:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.130195:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae548 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b0a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.130357:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660359168:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.130412:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.134800:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660359168:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.134845:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae548 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b0a expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.134871:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae548 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b0a expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.134913:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845041.134980:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641727104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4812us (5801us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.134989:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1007 00000100:00100000:3.0:1788845041.135457:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660359040:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845041.135475:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:3.0:1788845041.135488:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae541 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b03 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.135502:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.135505:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.135509:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae541 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b03 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.135521:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:3.0:1788845041.135559:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641726976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6035us (6225us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.135565:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1405 00000100:00100000:1.0:1788845041.138262:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.138307:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.138443:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641727232 00000100:00100000:3.0:1788845041.138453:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641727232 00000100:00100000:3.0:1788845041.138517:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1406 00000100:00100000:3.0:1788845041.138528:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641727232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.138551:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.138571:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287ae54f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b11 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.138623:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.138665:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660359296:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.138757:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.139348:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641727360 00000100:00100000:2.0:1788845041.139355:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641727360 00000100:00100000:2.0:1788845041.139394:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1008 00000100:00100000:2.0:1788845041.139403:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641727360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.139414:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.139427:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae556 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b18 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.139469:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660359424:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.139528:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.144104:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660359296:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845041.144134:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:3.0:1788845041.144156:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287ae54f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b11 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.144179:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.144182:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.144188:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287ae54f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b11 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.144203:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845041.144277:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641727232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5750us (5968us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.144285:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1406 00000100:00100000:2.0:1788845041.146280:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660359424:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.146316:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae556 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b18 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.146337:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae556 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b18 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.146349:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845041.146405:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641727360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7001us (8151us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.146412:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1008 00000100:00100000:1.0:1788845041.152152:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.152266:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641727488 00000100:00100000:0.0:1788845041.152278:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641727488 00002000:00100000:0.0:1788845041.152331:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d5880 x1875739641727488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.152343:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d5880 with x1875739641727488 00000100:00100000:0.0:1788845041.152376:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 934 00000100:00100000:0.0:1788845041.152385:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641727488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845041.152688:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641727488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 302us (544us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.152696:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 934 00002000:00100000:0.0:1788845041.152710:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d5880 x1875739641727488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.152719:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d5880 with x1875739641727488 00000100:00100000:1.0:1788845041.156143:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.156254:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641727616 00000100:00100000:0.0:1788845041.156264:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641727616 00002000:00100000:0.0:1788845041.156314:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d5500 x1875739641727616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.156326:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d5500 with x1875739641727616 00000100:00100000:0.0:1788845041.156363:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 935 00000100:00100000:0.0:1788845041.156373:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641727616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845041.156646:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641727616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 273us (512us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.156654:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 935 00002000:00100000:0.0:1788845041.156661:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d5500 x1875739641727616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.156669:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d5500 with x1875739641727616 00000100:00100000:1.0:1788845041.159996:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.160087:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641727744 00000100:00100000:0.0:1788845041.160097:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641727744 00002000:00100000:0.0:1788845041.160153:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d4380 x1875739641727744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.160164:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d4380 with x1875739641727744 00000100:00100000:0.0:1788845041.160205:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 936 00000100:00100000:0.0:1788845041.160214:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641727744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845041.160477:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641727744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 263us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.160484:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 936 00002000:00100000:0.0:1788845041.160491:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d4380 x1875739641727744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.160499:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d4380 with x1875739641727744 00000100:00100000:1.0:1788845041.164765:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.164990:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641727872 00000100:00100000:0.0:1788845041.165000:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641727872 00002000:00100000:0.0:1788845041.165048:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d6a00 x1875739641727872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.165058:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d6a00 with x1875739641727872 00000100:00100000:0.0:1788845041.165098:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 937 00000100:00100000:0.0:1788845041.165108:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641727872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845041.165801:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641727872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 691us (1043us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.165810:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 937 00002000:00100000:0.0:1788845041.165818:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d6a00 x1875739641727872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.165825:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d6a00 with x1875739641727872 00000100:00100000:1.0:1788845041.169440:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.169528:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641728000 00000100:00100000:0.0:1788845041.169538:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641728000 00002000:00100000:0.0:1788845041.169711:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d5180 x1875739641728000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.169728:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d5180 with x1875739641728000 00000100:00100000:0.0:1788845041.169819:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 938 00000100:00100000:0.0:1788845041.169830:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641728000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845041.170146:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641728000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 317us (716us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.170154:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 938 00002000:00100000:0.0:1788845041.170162:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359d5180 x1875739641728000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.170169:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359d5180 with x1875739641728000 00000100:00100000:1.0:1788845041.175610:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.175798:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641728128 00000100:00100000:3.0:1788845041.175808:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641728128 00000100:00100000:3.0:1788845041.175866:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1009 00000100:00100000:3.0:1788845041.175877:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641728128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.175929:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.175948:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b1f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.175980:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 14 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.175988:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(11 14) 00010000:00010000:3.0:1788845041.175992:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 14) 00010000:00010000:3.0:1788845041.175999:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1384448->1400831] (req 1384448->1400831) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.176036:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 14 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.176053:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 14 pid: 7983 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.176067:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660359552:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845041.176144:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.180755:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660359552:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845041.180797:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1384448->1400831] (req 1384448->1400831) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 10 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.180820:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845041.180926:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641728128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5050us (5323us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845041.180935:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1009 00000100:00100000:1.0:1788845041.183122:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.183233:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641728256 00000100:00100000:0.0:1788845041.183243:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641728256 00010000:00010000:0.0:1788845041.183299:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 15 pid: 7983 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.183345:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 487 00000100:00100000:0.0:1788845041.183355:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641728256:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845041.183371:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845041.183393:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 15 pid: 7983 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.183413:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 15 pid: 7983 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.183440:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae484 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1134592->1200127) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9a46 expref: 15 pid: 7983 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.183457:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 14) 00010000:00010000:0.0:1788845041.183461:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 14) 00010000:00010000:0.0:1788845041.183470:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.183480:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.183516:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.183529:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660359680:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845041.183632:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c300 going to sleep for 11 seconds 00000100:00080000:1.0:1788845041.184328:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845041.188953:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.189089:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660359680:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845041.189117:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845041.189167:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641728256:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5813us (6056us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.189178:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 487 00000100:00100000:2.0:1788845041.189460:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641728384 00000100:00100000:2.0:1788845041.189469:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641728384 00002000:00100000:2.0:1788845041.189514:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1384448->1388543) req@ffff949f03456a00 x1875739641728384/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.189525:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03456a00 with x1875739641728384 ext(1384448->1388543) 00002000:00010000:2.0:1788845041.189530:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03456a00 00000100:00100000:2.0:1788845041.189569:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 474 00000100:00100000:2.0:1788845041.189587:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03456a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641728384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845041.189823:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03456a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641728384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 245us (879us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845041.189831:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 474 00002000:00100000:2.0:1788845041.189837:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1384448->1388543) req@ffff949f03456a00 x1875739641728384/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/4536 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.189845:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03456a00 with x1875739641728384 ext(1384448->1388543) 00002000:00010000:2.0:1788845041.189848:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03456a00 00000100:00100000:1.0:1788845041.193862:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.194037:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641728512 00000100:00100000:2.0:1788845041.194047:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641728512 00002000:00100000:2.0:1788845041.194097:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1396736->1400831) req@ffff949f03454000 x1875739641728512/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.194109:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03454000 with x1875739641728512 ext(1396736->1400831) 00002000:00010000:2.0:1788845041.194114:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03454000 00000100:00100000:2.0:1788845041.194195:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 475 00000100:00100000:2.0:1788845041.194205:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641728512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845041.194486:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641728512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 282us (633us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845041.194494:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 475 00002000:00100000:2.0:1788845041.194501:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1396736->1400831) req@ffff949f03454000 x1875739641728512/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/4536 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.194509:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03454000 with x1875739641728512 ext(1396736->1400831) 00002000:00010000:2.0:1788845041.194513:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03454000 00000100:00100000:1.0:1788845041.199922:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.200003:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641728640 00000100:00100000:3.0:1788845041.200014:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641728640 00000100:00100000:3.0:1788845041.200075:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1010 00000100:00100000:3.0:1788845041.200086:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641728640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.200105:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.200122:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae564 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b26 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.200182:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660359808:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.200273:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.203637:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660359808:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.203680:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae564 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b26 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.203718:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae564 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b26 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.203735:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:2.0:1788845041.203805:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641728640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3719us (3921us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.203814:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1010 00000100:00100000:1.0:1788845041.207104:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.207204:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641728768 00000100:00100000:0.0:1788845041.207215:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641728768 00000100:00100000:0.0:1788845041.207295:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1011 00000100:00100000:0.0:1788845041.207306:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641728768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.207325:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.207346:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae56b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b2d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.207411:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660359936:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.207500:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.210975:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660359936:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.211015:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae56b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b2d expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.211039:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae56b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b2d expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.211054:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845041.211116:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641728768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3811us (4022us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.211125:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1011 00000100:00100000:1.0:1788845041.216209:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.216331:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641728896 00000100:00100000:2.0:1788845041.216343:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641728896 00000100:00100000:2.0:1788845041.216399:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1407 00000100:00100000:2.0:1788845041.216409:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641728896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.216432:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.216455:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b34 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.216489:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.216499:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b34 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.216534:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.216567:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 10 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.216588:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660360064:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845041.216676:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.221290:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.221409:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660360064:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845041.221451:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b34 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.221472:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.221476:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.221481:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b34 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.221493:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845041.221556:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641728896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5148us (5359us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.221565:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1407 00000100:00100000:0.0:1788845041.221647:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641729024 00000100:00100000:0.0:1788845041.221654:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641729024 00010000:00010000:0.0:1788845041.221684:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.221726:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 488 00000100:00100000:0.0:1788845041.221736:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d10050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641729024:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845041.221750:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845041.221775:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.221798:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.221820:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae502 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ac4 expref: 11 pid: 11036 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.221835:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.221840:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b34 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.221850:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b34 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.221859:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b34 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.221926:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b34 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.221941:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660360192:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845041.221996:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.230523:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035687100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660360192:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845041.230561:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845041.230624:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d10050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641729024:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8889us (9343us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.230639:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 488 00000100:00100000:1.0:1788845041.233084:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.233619:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.234111:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641729280 00000100:00100000:0.0:1788845041.234123:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641729280 00002000:00100000:0.0:1788845041.234177:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1470463) req@ffff949f03454700 x1875739641729280/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 504/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.234189:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03454700 with x1875739641729280 ext(1048576->1470463) 00002000:00010000:0.0:1788845041.234195:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03454700 00000100:00100000:0.0:1788845041.234238:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 476 00000100:00100000:0.0:1788845041.234248:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641729280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845041.236284:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641729152 00000100:00100000:2.0:1788845041.236296:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641729152 00002000:00100000:2.0:1788845041.236342:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03457480 x1875739641729152/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.236354:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03457480 with x1875739641729152 00000100:00100000:2.0:1788845041.236389:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 939 00000100:00100000:2.0:1788845041.236399:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641729152:12345-192.168.201.29@tcp:3:fsx.0 00010000:00010000:1.0:1788845041.239290:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845041.250440:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641729280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 16182us (16811us total) trans 0 rc 405504/405504 00000100:00100000:2.0:1788845041.250455:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 476 00002000:00100000:2.0:1788845041.250467:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1470463) req@ffff949f03454700 x1875739641729280/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 504/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 405504/405504 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.250478:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03454700 with x1875739641729280 ext(1048576->1470463) 00002000:00010000:2.0:1788845041.250484:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03454700 00000100:00100000:2.0:1788845041.252805:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641729152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 16397us (19721us total) trans 0 rc 413696/413696 00000100:00100000:2.0:1788845041.252819:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 939 00002000:00100000:2.0:1788845041.252832:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03457480 x1875739641729152/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 413696/413696 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.252841:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03457480 with x1875739641729152 00000100:00100000:2.0:1788845041.257769:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.257811:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.257903:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641729408 00000100:00100000:0.0:1788845041.257913:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641729408 00000100:00100000:0.0:1788845041.257972:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1408 00000100:00100000:0.0:1788845041.257982:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641729408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.258002:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.258021:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b3b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.258059:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845041.258071:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 411260 00010000:00010000:0.0:1788845041.258078:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.258092:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.258095:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845041.258099:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:3.0:1788845041.258190:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641729536 00000100:00100000:3.0:1788845041.258194:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641729536 00000100:00100000:3.0:1788845041.258242:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1012 00000100:00100000:3.0:1788845041.258250:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641729536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.258259:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.258270:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae580 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b42 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.258318:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660360320:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.258672:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641729408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 690us (912us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.258680:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1408 00000100:00100000:3.0:1788845041.258745:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e220c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.261612:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660360320:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845041.261654:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae580 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b42 expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.261679:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae580 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b42 expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.261692:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:3.0:1788845041.261750:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641729536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3501us (3938us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.261758:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1012 00000100:00100000:1.0:1788845041.265487:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.265574:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641729664 00000100:00100000:3.0:1788845041.265646:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641729664 00000100:00100000:3.0:1788845041.265760:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1013 00000100:00100000:3.0:1788845041.265771:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641729664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.265790:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.265807:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae587 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b49 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.265862:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660360448:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.265971:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e220c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.268543:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660360448:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.268649:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae587 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b49 expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.268673:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcf6428a/0xb2187301287ae587 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b49 expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.268688:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031a00, rc 0) 00000100:00100000:0.0:1788845041.268798:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641729664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3029us (3320us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.268807:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1013 00000100:00100000:2.0:1788845041.272856:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.273013:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641729792 00000100:00100000:3.0:1788845041.273023:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641729792 00000100:00100000:3.0:1788845041.273116:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1014 00000100:00100000:3.0:1788845041.273127:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641729792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.273145:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.273162:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae58e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b50 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.273350:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660360576:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.273472:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.276097:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660360576:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845041.276140:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae58e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b50 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.276166:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000418eaf48/0xb2187301287ae58e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b50 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.276186:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:3.0:1788845041.276403:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641729792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3277us (3557us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.276411:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1014 00000100:00100000:0.0:1788845041.279717:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.279794:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641729920 00000100:00100000:1.0:1788845041.279804:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641729920 00000100:00100000:1.0:1788845041.279866:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1015 00000100:00100000:1.0:1788845041.279876:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641729920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.279923:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.279942:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae595 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b57 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.280005:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660360704:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.280087:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.283296:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660360704:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.283334:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae595 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b57 expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.283358:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae595 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b57 expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.283370:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845041.283425:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641729920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3550us (3726us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.283434:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1015 00000100:00100000:3.0:1788845041.287642:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.287797:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641730048 00000100:00100000:2.0:1788845041.287807:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641730048 00000100:00100000:2.0:1788845041.287867:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1016 00000100:00100000:2.0:1788845041.287878:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641730048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.287944:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.287964:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae59c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b5e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.288026:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660360832:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.288112:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.291638:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660360832:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.291684:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae59c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b5e expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.291723:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f3731c1/0xb2187301287ae59c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b5e expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.291737:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845041.291811:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641730048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3935us (4178us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.291819:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1016 00000100:00100000:0.0:1788845041.295301:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.295509:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641730176 00000100:00100000:2.0:1788845041.295520:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641730176 00000100:00100000:2.0:1788845041.295590:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1017 00000100:00100000:2.0:1788845041.295600:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641730176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.295619:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.295635:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae5a3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b65 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.295694:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660360960:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.295777:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.298517:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660360960:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.298556:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae5a3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b65 expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.298592:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae5a3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b65 expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.298612:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845041.298677:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641730176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3077us (3384us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.298685:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1017 00000100:00100000:0.0:1788845041.302696:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.302790:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641730432 00000100:00100000:3.0:1788845041.302800:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641730432 00000100:00100000:3.0:1788845041.302849:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1409 00000100:00100000:3.0:1788845041.302860:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641730432:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845041.302947:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845041.302974:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae40d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99d6 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.302992:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.302997:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845041.303461:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.303477:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.303486:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.303520:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.303543:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 10 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.303558:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660361088:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845041.304195:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.306946:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660361088:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845041.306976:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.308738:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.308820:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641730560 00000100:00100000:2.0:1788845041.308829:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641730560 00010000:00010000:2.0:1788845041.308867:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 11 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.308935:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 489 00000100:00100000:2.0:1788845041.308945:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641730560:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845041.308960:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845041.308966:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 11 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.308983:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 11 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.309007:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae430 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a99f2 expref: 11 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.309022:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.309027:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.309036:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.309044:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.309071:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845041.309125:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641730560:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 181us (394us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.309132:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 489 00010000:00010000:3.0:1788845041.309632:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.309641:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845041.309762:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967802 00000004:00010000:3.0:1788845041.309814:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 411260 -> 411260 00000004:00010000:3.0:1788845041.309818:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:3.0:1788845041.309820:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845040 00000004:00010000:3.0:1788845041.309822:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:3.0:1788845041.309824:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:3.0:1788845041.309831:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000045bf556b reply state 00000000d48262bf transno 4294967803 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.309840:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845041.309945:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641730432:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 7087us (7259us total) trans 4294967803 rc 0/0 00000100:00100000:3.0:1788845041.309956:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1409 00000020:00080000:3.0:1788845041.309960:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845041.313423:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.313441:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.313450:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.313456:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02031c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845041.313461:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.313468:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.313484:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.313487:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845041.313493:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.313498:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287ae5aa lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.318322:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.318484:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641730688 00000100:00100000:3.0:1788845041.318494:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641730688 00000100:00100000:3.0:1788845041.318557:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1410 00000100:00100000:3.0:1788845041.318567:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641730688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.318600:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.318603:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845041.318650:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae572 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b34 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.318668:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.318672:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845041.318685:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b6c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.318719:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.318728:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.318762:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.318780:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 9 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.318799:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660361216:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845041.318923:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.322713:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660361216:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845041.322752:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.322772:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.322775:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.322779:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.322791:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031400, rc 0) 00000100:00100000:3.0:1788845041.322850:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641730688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4284us (4537us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845041.322859:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1410 00000100:00100000:2.0:1788845041.325615:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.325729:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641730816 00000100:00100000:1.0:1788845041.325739:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641730816 00010000:00010000:1.0:1788845041.325778:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 10 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.325823:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 490 00000100:00100000:1.0:1788845041.325833:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641730816:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845041.325848:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845041.325876:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 10 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.325924:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 10 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.325947:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae579 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b3b expref: 10 pid: 5840 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.325964:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.325971:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.325981:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.325994:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.326033:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.326046:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660361344:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845041.326133:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.329132:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660361344:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845041.329160:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845041.329235:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641730816:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3389us (3614us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845041.329247:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 490 00000100:00100000:2.0:1788845041.339131:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.339176:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.339374:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641730944 00000100:00100000:1.0:1788845041.339385:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641730944 00002000:00010000:1.0:1788845041.339426:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845041.339431:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f4380 with x1875739641730944 00000100:00100000:1.0:1788845041.339474:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 940 00000100:00100000:1.0:1788845041.339484:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641730944:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845041.339692:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 411260 -> 546573 00000004:00010000:1.0:1788845041.339695:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.339725:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845040 -> 1788845040 00000004:00010000:1.0:1788845041.339728:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.339731:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:1.0:1788845041.339802:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641730944:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 319us (624us total) trans 4294967804 rc 0/0 00000100:00100000:1.0:1788845041.339810:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 940 00002000:00010000:1.0:1788845041.339815:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845041.339817:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f033f4380 with x1875739641730944 00000100:00100000:0.0:1788845041.340870:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641731072 00000100:00100000:0.0:1788845041.340879:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641731072 00002000:00010000:0.0:1788845041.340950:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845041.340955:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f4000 with x1875739641731072 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845041.340965:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f4000. 00000100:00100000:0.0:1788845041.340989:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 477 00000100:00100000:0.0:1788845041.340998:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641731072:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845041.341814:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1469132 -> 0 00002000:00010000:0.0:1788845041.341822:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00002000:00010000:0.0:1788845041.341824:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845041 -> 1788845040 00002000:00010000:0.0:1788845041.341826:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00002000:00010000:0.0:1788845041.341829:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:0.0:1788845041.342421:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641731072:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 1420us (3297us total) trans 4294967474 rc 0/0 00000100:00100000:0.0:1788845041.342431:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 477 00002000:00010000:0.0:1788845041.342435:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845041.342438:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f4000 with x1875739641731072 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845041.342444:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f4000. 00000100:00100000:2.0:1788845041.346949:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.347217:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641731200 00000100:00100000:1.0:1788845041.347239:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641731200 00000100:00100000:1.0:1788845041.347298:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1411 00000100:00100000:1.0:1788845041.347309:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641731200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.347335:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.347356:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae5b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b73 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.347443:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae5bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.347457:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae5bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.347468:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae5bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845041.347496:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 546573 00000004:00200000:1.0:1788845041.347501:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845041.347519:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae5bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.347546:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae5b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b73 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.347568:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae5bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b73 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.348423:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.348429:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845041.348432:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845041.348665:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641731200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1356us (1724us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845041.348674:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1411 00000100:00100000:1.0:1788845041.354836:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.355003:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641731328 00000100:00100000:2.0:1788845041.355013:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641731328 00002000:00100000:2.0:1788845041.355062:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454380 x1875739641731328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.355073:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454380 with x1875739641731328 00000100:00100000:2.0:1788845041.355154:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 941 00000100:00100000:2.0:1788845041.355163:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641731328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845041.363250:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845041.363276:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02382a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660361472:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845041.363351:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845041.363452:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660361472 00000100:00100000:0.0:1788845041.363461:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660361472 00000100:00100000:0.0:1788845041.363513:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 83 00000100:00100000:0.0:1788845041.363522:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660361472:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788845041.363557:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 134545408 pending 3407872 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788845041.363611:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660361472:12345-0@lo:13:osp-pre-1-0.0 Request processed in 89us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.363618:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 83 00000100:00100000:3.0:1788845041.363914:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02382a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660361472:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845041.365857:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641731328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 10686us (11023us total) trans 0 rc 548864/548864 00000100:00100000:0.0:1788845041.365868:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 941 00002000:00100000:0.0:1788845041.365899:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03454380 x1875739641731328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 548864/548864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845041.365917:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03454380 with x1875739641731328 00000100:00100000:1.0:1788845041.370069:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.370155:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641731456 00000100:00100000:3.0:1788845041.370165:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641731456 00000100:00100000:3.0:1788845041.370241:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1412 00000100:00100000:3.0:1788845041.370252:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641731456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.370274:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.370294:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae5c6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b7a expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.370384:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.370399:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.370411:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845041.370437:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 546573 00000004:00200000:3.0:1788845041.370441:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845041.370460:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.370486:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae5c6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.370504:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.370512:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.370515:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845041.370518:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845041.370585:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641731456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 334us (525us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845041.370592:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1412 00000100:00100000:3.0:1788845041.373874:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.374113:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641731584 00000100:00100000:0.0:1788845041.374124:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641731584 00000100:00100000:0.0:1788845041.374186:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1413 00000100:00100000:0.0:1788845041.374197:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641731584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.374218:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.374361:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae5d4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b81 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.374399:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.374442:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660361600:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.374529:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.376627:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.376866:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641731712 00000100:00100000:3.0:1788845041.376876:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641731712 00000100:00100000:3.0:1788845041.376960:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1018 00000100:00100000:3.0:1788845041.376971:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641731712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.376990:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.377008:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae5db lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b88 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.377063:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660361728:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.377138:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000004:00010000:0.0:1788845041.379343:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845040 -> 1788845041 00000100:00100000:0.0:1788845041.379379:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660361600:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845041.379402:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:0.0:1788845041.379422:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae5d4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b81 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.379440:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.379443:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845041.379448:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae5d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b81 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.379462:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845041.379517:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641731584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5321us (5652us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.379525:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1413 00002000:00010000:3.0:1788845041.381948:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845040 -> 1788845041 00000100:00100000:3.0:1788845041.381981:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660361728:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845041.382018:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae5db lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b88 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.382043:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287ae5db lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b88 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.382057:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:3.0:1788845041.382115:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641731712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5145us (5496us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.382123:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1018 00000100:00100000:2.0:1788845041.385980:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.386018:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.386115:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641731840 00000100:00100000:3.0:1788845041.386125:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641731840 00000100:00100000:3.0:1788845041.386175:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1414 00000100:00100000:3.0:1788845041.386187:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641731840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.386207:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.386239:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae5e2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b8f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.386288:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.386330:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660361856:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.386420:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.386840:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641731968 00000100:00100000:1.0:1788845041.386845:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641731968 00000100:00100000:1.0:1788845041.386914:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1019 00000100:00100000:1.0:1788845041.386924:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641731968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.386933:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.386945:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae5e9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b96 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.386985:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03455180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660361984:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.387032:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.390932:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03455180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660361984:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.390975:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae5e9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b96 expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.391005:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021600b89/0xb2187301287ae5e9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b96 expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.391019:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845041.391077:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641731968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4155us (5057us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.391086:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1019 00000100:00100000:3.0:1788845041.391557:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660361856:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845041.391574:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:3.0:1788845041.391588:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae5e2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b8f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.391601:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.391604:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.391609:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae5e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b8f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.391620:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:3.0:1788845041.391714:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641731840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5529us (5742us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.391721:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1414 00000100:00100000:2.0:1788845041.398063:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.398135:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641732096 00000100:00100000:1.0:1788845041.398145:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641732096 00000100:00100000:1.0:1788845041.398200:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1415 00000100:00100000:1.0:1788845041.398209:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641732096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.398242:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.398262:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae5f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9b9d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.398301:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.398341:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03455180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660362112:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.398422:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.401162:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.401245:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641732224 00000100:00100000:3.0:1788845041.401264:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641732224 00000100:00100000:3.0:1788845041.401325:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1020 00000100:00100000:3.0:1788845041.401337:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641732224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.401358:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.401377:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae5f7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ba4 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.401437:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660362240:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.401793:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.407014:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03455180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660362112:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845041.407042:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:0.0:1788845041.407061:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae5f0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b9d expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.407085:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.407088:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845041.407093:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae5f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b9d expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.407107:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:0.0:1788845041.407172:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641732096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8963us (9117us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.407181:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1415 00000100:00100000:3.0:1788845041.411505:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660362240:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845041.411552:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae5f7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ba4 expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.411652:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287ae5f7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ba4 expref: 15 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.411670:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:3.0:1788845041.411784:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641732224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10447us (10630us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.411793:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1020 00000100:00100000:1.0:1788845041.415843:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.415878:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.415977:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641732480 00000100:00100000:2.0:1788845041.415989:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641732480 00000100:00100000:2.0:1788845041.416050:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1021 00000100:00100000:2.0:1788845041.416061:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641732480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.416083:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.416103:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b0876318/0xb2187301287ae5fe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bb2 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.416168:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660362368:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.416296:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641732352 00000100:00100000:3.0:1788845041.416301:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641732352 00000100:00100000:3.0:1788845041.416335:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1416 00000100:00100000:3.0:1788845041.416342:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641732352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.416353:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.416365:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae605 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bab expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.416401:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.416426:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660362496:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.416958:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.417039:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.420250:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660362496:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845041.420279:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:3.0:1788845041.420299:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae605 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bab expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.420320:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.420323:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.420328:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae605 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bab expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.420344:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:3.0:1788845041.420396:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641732352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4053us (4515us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.420404:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1416 00000100:00100000:0.0:1788845041.420636:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660362368:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.420665:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b0876318/0xb2187301287ae5fe lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bb2 expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.420685:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b0876318/0xb2187301287ae5fe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bb2 expref: 15 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.420695:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032600, rc 0) 00000100:00100000:0.0:1788845041.420733:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641732480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4675us (4900us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.420740:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1021 00000100:00100000:0.0:1788845041.425810:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.425952:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.426064:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641732608 00000100:00100000:3.0:1788845041.426074:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641732608 00000100:00100000:3.0:1788845041.426134:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1417 00000100:00100000:3.0:1788845041.426145:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641732608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.426166:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.426186:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae60c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bb9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.426463:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.426509:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660362624:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.426714:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.426774:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641732736 00000100:00100000:2.0:1788845041.426780:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641732736 00000100:00100000:2.0:1788845041.426847:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1022 00000100:00100000:2.0:1788845041.426856:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641732736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.426866:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.426877:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae613 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bc0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.426935:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660362752:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.426964:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.434436:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660362752:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.434484:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae613 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bc0 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.434510:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae613 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bc0 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.434524:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:0.0:1788845041.434599:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2368:x1875739641732736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7743us (8797us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.434608:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1022 00000100:00100000:3.0:1788845041.434836:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660362624:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845041.434853:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:3.0:1788845041.434865:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae60c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bb9 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.434879:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.434926:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.434931:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae60c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bb9 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.434940:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:3.0:1788845041.434966:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641732608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8824us (9014us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.434973:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1417 00000100:00100000:1.0:1788845041.439199:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.439257:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.439436:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641732864 00000100:00100000:3.0:1788845041.439445:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641732864 00000100:00100000:3.0:1788845041.439513:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1418 00000100:00100000:3.0:1788845041.439523:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641732864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.439544:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.439565:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae61a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bc7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.439611:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.439653:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660362880:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845041.439761:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.439966:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641732992 00000100:00100000:0.0:1788845041.439970:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641732992 00000100:00100000:0.0:1788845041.439999:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1023 00000100:00100000:0.0:1788845041.440007:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641732992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.440016:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.440027:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287ae621 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bce expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.440061:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660363008:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.440091:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.443925:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660362880:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845041.443953:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:3.0:1788845041.443973:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae61a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bc7 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.443992:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.443994:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.443999:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae61a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bc7 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.444013:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:3.0:1788845041.444079:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641732864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4557us (4890us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.444086:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1418 00000100:00100000:0.0:1788845041.446241:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660363008:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.446288:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287ae621 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bce expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.446311:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287ae621 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bce expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.446323:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845041.446382:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641732992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6375us (7124us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.446390:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1023 00000100:00100000:1.0:1788845041.450459:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.450536:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641733248 00000100:00100000:0.0:1788845041.450545:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641733248 00000100:00100000:0.0:1788845041.450605:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1419 00000100:00100000:0.0:1788845041.450615:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641733248:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845041.450667:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845041.450693:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae5bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b73 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.450718:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.450723:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845041.450944:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.450958:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.450971:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.450999:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.451016:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 9 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.451029:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7e300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660363136:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845041.451103:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.460339:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.460851:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641733376 00000100:00100000:1.0:1788845041.460860:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641733376 00010000:00010000:1.0:1788845041.460923:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 10 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.460949:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 491 00000100:00100000:1.0:1788845041.460956:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641733376:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845041.460967:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845041.460972:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 10 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.460985:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 10 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.461003:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.461006:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.461014:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.461019:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.461031:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845041.461076:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641733376:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 120us (745us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845041.461083:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 491 00010000:00010000:0.0:1788845041.461278:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287ae5cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b7a expref: 9 pid: 5839 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.461297:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7e300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660363136:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845041.461316:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.461325:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.461331:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845041.461841:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967805 00000004:00010000:1.0:1788845041.463773:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 546573 -> 546573 00000004:00010000:1.0:1788845041.463782:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.463785:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845040 00000004:00010000:1.0:1788845041.463787:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.463789:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:1.0:1788845041.463803:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003c1c2c97 reply state 00000000c5ae92fd transno 4294967806 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.463814:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845041.464097:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641733248:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13479us (13643us total) trans 4294967806 rc 0/0 00000100:00100000:1.0:1788845041.464105:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1419 00000020:00080000:1.0:1788845041.464110:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845041.467042:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.467068:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.467077:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.467084:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845041.467091:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.467102:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.467124:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.467128:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845041.467133:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.467140:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae628 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.468028:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.470017:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641733504 00000100:00100000:2.0:1788845041.470031:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641733504 00002000:00010000:2.0:1788845041.470077:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1970627->18446744073709551615) 00002000:00100000:2.0:1788845041.470081:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365180 with x1875739641733504 ext(1970627->18446744073709551615) 00002000:00010000:2.0:1788845041.470094:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365180. 00000100:00100000:2.0:1788845041.470123:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 478 00000100:00100000:2.0:1788845041.470132:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641733504:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845041.471067:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1970627 00002000:00010000:2.0:1788845041.471073:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00002000:00010000:2.0:1788845041.471076:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845041 -> 1788845040 00002000:00010000:2.0:1788845041.471078:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00002000:00010000:2.0:1788845041.471080:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 293 00000100:00100000:2.0:1788845041.471386:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641733504:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 1251us (3359us total) trans 4294967475 rc 0/0 00000100:00100000:2.0:1788845041.471395:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 478 00002000:00010000:2.0:1788845041.471400:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1970627->18446744073709551615) 00002000:00100000:2.0:1788845041.471403:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365180 with x1875739641733504 ext(1970627->18446744073709551615) 00002000:00010000:2.0:1788845041.471409:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365180. 00000100:00100000:2.0:1788845041.481675:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.481776:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.481873:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641733760 00000100:00100000:1.0:1788845041.481920:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641733760 00002000:00100000:1.0:1788845041.481968:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358fb80 x1875739641733760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845041.481979:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358fb80 with x1875739641733760 00000100:00100000:1.0:1788845041.482016:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 942 00000100:00100000:1.0:1788845041.482025:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641733760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845041.483095:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641733632 00000100:00100000:3.0:1788845041.483104:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641733632 00002000:00100000:2.0:1788845041.483338:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1974271) req@ffff949f033f2a00 x1875739641733632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.483349:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f2a00 with x1875739641733632 ext(1048576->1974271) 00002000:00010000:2.0:1788845041.483355:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f2a00 00000100:00100000:2.0:1788845041.483418:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 479 00000100:00100000:2.0:1788845041.483427:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641733632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845041.486711:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:2368:x1875739641733760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4666us (4914us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845041.486725:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 942 00002000:00100000:1.0:1788845041.486739:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0358fb80 x1875739641733760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845041.486748:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358fb80 with x1875739641733760 00000100:00100000:2.0:1788845041.520318:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641733632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 36881us (38641us total) trans 0 rc 925696/925696 00000100:00100000:2.0:1788845041.520334:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 479 00002000:00100000:2.0:1788845041.520348:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1974271) req@ffff949f033f2a00 x1875739641733632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 925696/925696 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.520358:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f2a00 with x1875739641733632 ext(1048576->1974271) 00002000:00010000:2.0:1788845041.520366:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f2a00 00000100:00100000:1.0:1788845041.525523:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.525683:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641733888 00000100:00100000:2.0:1788845041.525694:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641733888 00002000:00100000:2.0:1788845041.525751:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1949696->1970626) req@ffff94a0359ba680 x1875739641733888/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.525763:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359ba680 with x1875739641733888 ext(1949696->1970626) 00002000:00010000:2.0:1788845041.525769:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359ba680 00000100:00100000:2.0:1788845041.525841:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 480 00000100:00100000:2.0:1788845041.525852:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:2368:x1875739641733888:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845041.530328:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641733888:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4467us (4802us total) trans 4294967476 rc 0/0 00000100:00100000:3.0:1788845041.530344:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 480 00002000:00100000:3.0:1788845041.530357:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1949696->1970626) req@ffff94a0359ba680 x1875739641733888/t4294967476(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/448 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845041.530367:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359ba680 with x1875739641733888 ext(1949696->1970626) 00002000:00010000:3.0:1788845041.530373:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359ba680 00000100:00100000:0.0:1788845041.534240:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.534402:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641734016 00000100:00100000:3.0:1788845041.534412:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641734016 00000100:00100000:3.0:1788845041.534509:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1024 00000100:00100000:3.0:1788845041.534521:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641734016:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845041.568533:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967474 is committed 00000001:00080000:1.0:1788845041.568553:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967466 is committed 00000001:00080000:1.0:1788845041.569389:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967475 is committed 00000001:00080000:1.0:1788845041.569398:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967473 is committed 00000100:00100000:0.0:1788845041.569622:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641734016:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 35099us (35395us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.569633:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1024 00000100:00100000:3.0:1788845041.572399:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.572564:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641734144 00000100:00100000:0.0:1788845041.572589:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641734144 00000100:00100000:0.0:1788845041.572652:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1420 00000100:00100000:0.0:1788845041.572663:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641734144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.572689:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.572720:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae62f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bd5 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.572816:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae636 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.572832:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae636 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.572846:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae636 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845041.572876:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1970627 00000004:00200000:0.0:1788845041.572920:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845041.572945:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae636 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.572975:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae62f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bd5 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.572994:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae636 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bd5 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.573003:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.573006:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845041.573009:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:0.0:1788845041.573069:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641734144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 408us (679us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.573077:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1420 00000001:00080000:1.0:1788845041.576957:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967476 is committed 00000100:00100000:2.0:1788845041.579767:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.579852:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641734272 00000100:00100000:3.0:1788845041.579862:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641734272 00000100:00100000:3.0:1788845041.579957:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1421 00000100:00100000:3.0:1788845041.579968:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641734272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.579989:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.580010:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae63d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bdc expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.580106:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.580121:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.580133:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845041.580163:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1970627 00000004:00200000:3.0:1788845041.580169:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845041.580191:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.580218:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae63d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.580245:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.580253:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.580257:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845041.580260:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:3.0:1788845041.580323:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641734272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 356us (566us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845041.580331:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1421 00000100:00100000:2.0:1788845041.583981:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.584020:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.584100:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641734528 00000100:00100000:1.0:1788845041.584110:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641734528 00000100:00100000:1.0:1788845041.584185:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1025 00000100:00100000:1.0:1788845041.584197:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641734528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.584214:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.584234:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ae64b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bea expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.584299:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660363264:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.584399:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.584558:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641734400 00000100:00100000:0.0:1788845041.584564:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641734400 00000100:00100000:0.0:1788845041.584609:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1422 00000100:00100000:0.0:1788845041.584617:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641734400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.584628:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.584641:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae652 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9be3 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.584670:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.584706:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660363392:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.584736:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6e40 going to sleep for 11 seconds 00002000:00010000:1.0:1788845041.587712:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845040 -> 1788845041 00000100:00100000:1.0:1788845041.587744:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660363264:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.587779:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ae64b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bea expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.587802:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090ef21e9/0xb2187301287ae64b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bea expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.587813:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845041.587874:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641734528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3678us (3852us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.587912:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1025 00000004:00010000:2.0:1788845041.588039:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845040 -> 1788845041 00000100:00100000:2.0:1788845041.588063:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660363392:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845041.588077:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:2.0:1788845041.588091:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae652 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9be3 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.588104:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.588108:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.588113:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae652 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9be3 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.588124:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:2.0:1788845041.588160:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641734400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3545us (4194us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.588172:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1422 00000100:00100000:3.0:1788845041.592056:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.592132:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.592247:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641734784 00000100:00100000:2.0:1788845041.592267:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641734784 00000100:00100000:2.0:1788845041.592322:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1026 00000100:00100000:2.0:1788845041.592333:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641734784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.592354:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.592374:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae659 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bf8 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.592436:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660363520:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.592521:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.593088:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641734656 00000100:00100000:0.0:1788845041.593094:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641734656 00000100:00100000:0.0:1788845041.593148:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1423 00000100:00100000:0.0:1788845041.593157:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641734656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.593170:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.593184:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae660 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bf1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.593213:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.593241:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660363648:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.593302:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.596834:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660363520:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.596876:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae659 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bf8 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.596932:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b47f006/0xb2187301287ae659 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bf8 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.596947:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032000, rc 0) 00000100:00100000:2.0:1788845041.596997:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641734784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4666us (4866us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.597005:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1026 00000100:00100000:0.0:1788845041.600832:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660363648:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845041.600860:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:0.0:1788845041.600881:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae660 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bf1 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.600916:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.600919:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845041.600925:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae660 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bf1 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.600943:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845041.601021:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641734656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7864us (8977us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.601029:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1423 00000100:00100000:3.0:1788845041.604343:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.604394:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.604468:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641735040 00000100:00100000:0.0:1788845041.604477:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641735040 00000100:00100000:0.0:1788845041.604536:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1027 00000100:00100000:0.0:1788845041.604547:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641735040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.604566:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.604596:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae667 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c06 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.604656:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660363776:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.604747:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.605507:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641734912 00000100:00100000:1.0:1788845041.605515:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641734912 00000100:00100000:1.0:1788845041.605558:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1424 00000100:00100000:1.0:1788845041.605566:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641734912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.605591:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.605607:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ae66e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9bff expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.605636:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.605663:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660363904:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.605742:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.609191:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660363776:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.609240:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae667 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c06 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.609283:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae667 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c06 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.609297:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845041.609358:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641735040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4813us (5023us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.609366:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1027 00000100:00100000:1.0:1788845041.609736:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660363904:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845041.609751:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:1.0:1788845041.609765:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ae66e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bff expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.609778:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.609781:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845041.609785:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15c00/0xb2187301287ae66e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bff expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.609795:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845041.609834:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641734912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4268us (5440us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.609842:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1424 00000100:00100000:3.0:1788845041.615277:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.615343:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.615420:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641735168 00000100:00100000:1.0:1788845041.615431:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641735168 00000100:00100000:1.0:1788845041.615490:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1425 00000100:00100000:1.0:1788845041.615499:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641735168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.615530:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.615552:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae675 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c0d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.615599:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.615639:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660364032:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.615733:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.616187:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641735296 00000100:00100000:0.0:1788845041.616193:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641735296 00000100:00100000:0.0:1788845041.616226:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1028 00000100:00100000:0.0:1788845041.616234:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641735296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.616245:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.616257:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae67c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c14 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.616295:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660364160:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.616331:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.623778:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660364160:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.623822:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae67c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c14 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.623846:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae67c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c14 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.623865:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845041.623953:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641735296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7719us (8609us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.623961:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1028 00000100:00100000:1.0:1788845041.624082:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359baa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660364032:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845041.624096:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:1.0:1788845041.624108:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae675 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c0d expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.624120:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.624123:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845041.624127:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae675 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c0d expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.624136:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15000, rc 0) 00000100:00100000:1.0:1788845041.624173:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641735168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8675us (8915us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.624181:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1425 00000100:00100000:2.0:1788845041.631626:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.631721:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641735552 00000100:00100000:0.0:1788845041.631732:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641735552 00000100:00100000:0.0:1788845041.631792:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1426 00000100:00100000:0.0:1788845041.631803:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641735552:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845041.631863:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845041.631927:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae636 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bd5 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.631946:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.631950:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845041.632189:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.632205:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.632212:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.632244:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.632276:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 9 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.632290:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660364288:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845041.632370:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.635519:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660364288:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845041.635550:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.636963:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.637627:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641735680 00000100:00100000:0.0:1788845041.637638:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641735680 00010000:00010000:0.0:1788845041.637684:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 10 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.637728:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 492 00000100:00100000:0.0:1788845041.637739:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641735680:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845041.637754:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845041.637761:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 10 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.637780:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 10 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.637801:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287ae644 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9bdc expref: 10 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.637825:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.637831:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.637841:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.637850:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.637914:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845041.637966:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641735680:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 228us (1010us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.637974:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 492 00010000:00010000:2.0:1788845041.638128:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.638137:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845041.638291:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967807 00000004:00010000:2.0:1788845041.638351:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 546573 -> 546573 00000004:00010000:2.0:1788845041.638355:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:2.0:1788845041.638357:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845040 00000004:00010000:2.0:1788845041.638360:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:2.0:1788845041.638362:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:2.0:1788845041.638369:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007ab5126f reply state 0000000009f122cb transno 4294967808 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.638378:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845041.638453:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+22:14891:x1875739641735552:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6652us (6836us total) trans 4294967808 rc 0/0 00000100:00100000:2.0:1788845041.638460:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1426 00000020:00080000:2.0:1788845041.638464:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788845041.644577:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:1.0:1788845041.644669:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.644683:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.644690:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.644725:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845041.644730:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.644738:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.644755:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.644758:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845041.644762:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.644768:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae683 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.645966:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641735808 00000100:00100000:0.0:1788845041.645981:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641735808 00002000:00010000:0.0:1788845041.646036:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (2026308->18446744073709551615) 00002000:00100000:0.0:1788845041.646041:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358d500 with x1875739641735808 ext(2026308->18446744073709551615) 00002000:00010000:0.0:1788845041.646051:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358d500. 00000100:00100000:0.0:1788845041.646149:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 481 00000100:00100000:0.0:1788845041.646161:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641735808:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845041.646471:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1970627 -> 2026308 00002000:00010000:0.0:1788845041.646474:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00002000:00010000:0.0:1788845041.646477:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00002000:00010000:0.0:1788845041.646479:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00002000:00010000:0.0:1788845041.646481:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 293 -> 57 00000100:00100000:0.0:1788845041.646670:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641735808:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 511us (2108us total) trans 4294967477 rc 0/0 00000100:00100000:0.0:1788845041.646678:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 481 00002000:00010000:0.0:1788845041.646681:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (2026308->18446744073709551615) 00002000:00100000:0.0:1788845041.646684:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358d500 with x1875739641735808 ext(2026308->18446744073709551615) 00002000:00010000:0.0:1788845041.646689:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358d500. 00000100:00100000:3.0:1788845041.652066:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.652283:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641735936 00000100:00100000:2.0:1788845041.652294:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641735936 00002000:00100000:2.0:1788845041.652348:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1974272->2027519) req@ffff94a03592a680 x1875739641735936/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.652360:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03592a680 with x1875739641735936 ext(1974272->2027519) 00002000:00010000:2.0:1788845041.652365:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03592a680 00000100:00100000:2.0:1788845041.652399:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 482 00000100:00100000:2.0:1788845041.652410:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641735936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845041.656132:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641735936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 3714us (4067us total) trans 0 rc 53248/53248 00000100:00100000:2.0:1788845041.656156:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 482 00002000:00100000:2.0:1788845041.656170:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1974272->2027519) req@ffff94a03592a680 x1875739641735936/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 53248/53248 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.656180:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03592a680 with x1875739641735936 ext(1974272->2027519) 00002000:00010000:2.0:1788845041.656187:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03592a680 00000100:00100000:1.0:1788845041.668160:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.668371:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641736064 00000100:00100000:2.0:1788845041.668382:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641736064 00002000:00100000:2.0:1788845041.668437:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1974272->2026307) req@ffff94a0359baa00 x1875739641736064/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.668449:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359baa00 with x1875739641736064 ext(1974272->2026307) 00002000:00010000:2.0:1788845041.668455:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359baa00 00000100:00100000:2.0:1788845041.668548:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 483 00000100:00100000:2.0:1788845041.668560:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641736064:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845041.672792:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641736064:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4223us (4642us total) trans 4294967478 rc 0/0 00000100:00100000:2.0:1788845041.672807:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 483 00002000:00100000:2.0:1788845041.672819:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1974272->2026307) req@ffff94a0359baa00 x1875739641736064/t4294967478(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 488/448 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.672830:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0359baa00 with x1875739641736064 ext(1974272->2026307) 00002000:00010000:2.0:1788845041.672837:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0359baa00 00000100:00100000:1.0:1788845041.677548:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.677708:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641736192 00000100:00100000:3.0:1788845041.677720:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641736192 00000100:00100000:3.0:1788845041.677835:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1029 00000100:00100000:3.0:1788845041.677847:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641736192:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:0.0:1788845041.709195:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641736192:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 31341us (31645us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.709210:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1029 00000001:00080000:0.0:1788845041.709689:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967477 is committed 00000001:00080000:0.0:1788845041.709712:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967478 is committed 00000100:00100000:3.0:1788845041.712488:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.712773:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641736320 00000100:00100000:0.0:1788845041.712784:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641736320 00000100:00100000:0.0:1788845041.712906:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1427 00000100:00100000:0.0:1788845041.712919:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641736320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.712943:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.712965:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae68a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c1b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.713052:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.713067:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.713078:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845041.713107:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2026308 00000004:00200000:0.0:1788845041.713111:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845041.713130:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.713178:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae68a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.713195:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.713204:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.713207:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845041.713211:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845041.713394:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641736320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 478us (914us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.713401:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1427 00000100:00100000:0.0:1788845041.717847:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.718023:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641736448 00000100:00100000:1.0:1788845041.718033:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641736448 00000100:00100000:1.0:1788845041.718095:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1428 00000100:00100000:1.0:1788845041.718105:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641736448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.718129:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.718431:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae698 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c22 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.718475:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.718518:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660364416:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.718616:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.722075:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.722444:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641736576 00000100:00100000:1.0:1788845041.722455:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641736576 00000100:00100000:1.0:1788845041.722512:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1030 00000100:00100000:1.0:1788845041.722522:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641736576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.722541:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.722559:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ae69f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c29 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.722634:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660364544:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.722728:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000004:00010000:1.0:1788845041.725873:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845040 -> 1788845041 00000100:00100000:1.0:1788845041.725945:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660364416:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845041.725970:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:1.0:1788845041.725991:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae698 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c22 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.726013:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.726017:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845041.726023:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae698 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c22 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.726037:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845041.726103:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641736448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8000us (8265us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.726112:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1428 00002000:00010000:1.0:1788845041.727104:0:6682:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 57 -> 163 00000100:00100000:1.0:1788845041.727128:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660364544:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.727444:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ae69f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c29 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.727466:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287ae69f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c29 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.727478:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845041.727528:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641736576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5007us (5462us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.727535:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1030 00000100:00100000:3.0:1788845041.731411:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.731454:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.731591:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641736704 00000100:00100000:2.0:1788845041.731600:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641736704 00000100:00100000:2.0:1788845041.731662:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1429 00000100:00100000:2.0:1788845041.731673:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641736704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.731703:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.731725:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae6a6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c30 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.731761:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.731803:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660364672:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.731906:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.735120:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660364672:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845041.735152:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:2.0:1788845041.735174:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae6a6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c30 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.735196:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.735200:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.735205:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae6a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c30 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.735236:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:2.0:1788845041.735296:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641736704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3624us (3894us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.735306:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1429 00000100:00100000:0.0:1788845041.736828:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641736832 00000100:00100000:0.0:1788845041.736842:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641736832 00000100:00100000:0.0:1788845041.736997:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1031 00000100:00100000:0.0:1788845041.737007:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641736832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.737027:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.737045:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae6ad lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c37 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.737108:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660364800:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.737451:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.741510:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660364800:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.741554:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae6ad lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c37 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.741651:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae6ad lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c37 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.741666:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845041.741778:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641736832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4772us (10322us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.741787:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1031 00000100:00100000:3.0:1788845041.746543:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.746795:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.746997:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641737088 00000100:00100000:1.0:1788845041.747009:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641737088 00000100:00100000:1.0:1788845041.747068:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1032 00000100:00100000:1.0:1788845041.747077:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641737088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.747098:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.747118:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ae6b4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c45 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.747179:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660364928:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.747278:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.749944:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660364928:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.749986:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ae6b4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c45 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.750010:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ae6b4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c45 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.750022:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845041.750080:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641737088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3003us (3284us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.750089:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1032 00000100:00100000:0.0:1788845041.751977:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641736960 00000100:00100000:0.0:1788845041.751988:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641736960 00000100:00100000:2.0:1788845041.752205:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1430 00000100:00100000:2.0:1788845041.752554:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641736960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.752688:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.752718:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae6bb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c3e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.752760:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.752817:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660365056:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.753243:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.757524:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660365056:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845041.757556:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:0.0:1788845041.757598:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae6bb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c3e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.757618:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.757621:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845041.757625:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032400/0xb2187301287ae6bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c3e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.757639:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:0.0:1788845041.757711:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641736960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5171us (11180us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.757723:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1430 00000100:00100000:3.0:1788845041.760802:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.760848:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.760967:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641737216 00000100:00100000:0.0:1788845041.760977:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641737216 00000100:00100000:0.0:1788845041.761028:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1431 00000100:00100000:0.0:1788845041.761038:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035928e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641737216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.761062:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.761083:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae6c2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c4c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.761122:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.761164:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660365184:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.761262:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d69c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.761375:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641737344 00000100:00100000:1.0:1788845041.761380:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641737344 00000100:00100000:1.0:1788845041.761417:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1033 00000100:00100000:1.0:1788845041.761423:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641737344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.761432:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.761443:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ae6c9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c53 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.761478:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660365312:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.761505:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.765517:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660365312:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.765562:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ae6c9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c53 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.765596:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008213b370/0xb2187301287ae6c9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c53 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.765609:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845041.765681:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641737344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4257us (4832us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.765690:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1033 00000100:00100000:2.0:1788845041.766631:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660365184:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845041.766651:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 546573 00010000:00010000:2.0:1788845041.766667:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae6c2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c4c expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.766683:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.766686:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.766690:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae6c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c4c expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.766713:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:2.0:1788845041.766760:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035928e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641737216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5723us (5969us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.766768:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1431 00000100:00100000:0.0:1788845041.772775:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.773171:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641737600 00000100:00100000:1.0:1788845041.773182:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641737600 00000100:00100000:1.0:1788845041.773369:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1432 00000100:00100000:1.0:1788845041.773382:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641737600:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845041.773671:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.773690:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.773710:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.773743:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.773767:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 9 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.773781:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660365440:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845041.773918:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.779304:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.779378:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641737728 00000100:00100000:3.0:1788845041.779389:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641737728 00010000:00010000:3.0:1788845041.779431:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 10 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.779472:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 493 00000100:00100000:3.0:1788845041.779482:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641737728:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845041.779496:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845041.779503:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 10 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.779520:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 10 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.779543:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.779548:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.779559:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.779566:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.779596:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845041.779653:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641737728:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 173us (358us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845041.779660:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 493 00010000:00010000:1.0:1788845041.780988:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae691 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c1b expref: 9 pid: 5841 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.781012:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660365440:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845041.781036:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.781046:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.781051:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845041.781182:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967809 00000004:00010000:1.0:1788845041.781344:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 546573 -> 546573 00000004:00010000:1.0:1788845041.781348:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.781351:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845040 00000004:00010000:1.0:1788845041.781354:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.781356:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:1.0:1788845041.781363:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000847e40c2 reply state 000000004b1c1f18 transno 4294967810 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.781376:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845041.781529:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641737600:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 8149us (8762us total) trans 4294967810 rc 0/0 00000100:00100000:1.0:1788845041.781537:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1432 00000020:00080000:1.0:1788845041.781542:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788845041.784876:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.784940:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:1.0:1788845041.785020:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.785033:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.785040:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.785046:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d15800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845041.785051:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.785059:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.785077:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.785080:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845041.785085:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.785091:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287ae6d0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.785170:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641737856 00000100:00100000:0.0:1788845041.785180:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641737856 00002000:00010000:0.0:1788845041.785228:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845041.785233:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358c000 with x1875739641737856 00000100:00100000:0.0:1788845041.785259:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 943 00000100:00100000:0.0:1788845041.785270:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641737856:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845041.785552:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 546573 -> 358029 00000004:00010000:0.0:1788845041.785557:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.785559:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845040 -> 1788845041 00000004:00010000:0.0:1788845041.785562:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.785565:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:0.0:1788845041.785653:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641737856:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 384us (786us total) trans 4294967811 rc 0/0 00000100:00100000:0.0:1788845041.785660:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 943 00002000:00010000:0.0:1788845041.785664:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845041.785666:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0358c000 with x1875739641737856 00000100:00100000:3.0:1788845041.786672:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641737984 00000100:00100000:3.0:1788845041.786682:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641737984 00002000:00010000:3.0:1788845041.786724:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845041.786729:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358e300 with x1875739641737984 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845041.786740:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358e300. 00000100:00100000:3.0:1788845041.786773:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 484 00000100:00100000:3.0:1788845041.786782:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641737984:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845041.787185:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2026308 -> 0 00002000:00010000:3.0:1788845041.787189:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.787191:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.787194:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.787196:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 163 -> 163 00000100:00100000:3.0:1788845041.787249:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641737984:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 469us (2308us total) trans 4294967479 rc 0/0 00000100:00100000:3.0:1788845041.787257:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 484 00002000:00010000:3.0:1788845041.787261:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845041.787264:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358e300 with x1875739641737984 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845041.787268:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358e300. 00000100:00100000:2.0:1788845041.793864:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.794044:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641738112 00000100:00100000:1.0:1788845041.794055:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641738112 00000100:00100000:1.0:1788845041.794146:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1433 00000100:00100000:1.0:1788845041.794155:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641738112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.794177:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.794198:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae6d7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c5a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.794305:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae6de lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.794320:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae6de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.794331:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae6de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845041.794358:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 358029 00000004:00200000:1.0:1788845041.794362:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845041.794380:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae6de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.794404:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287ae6d7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c5a expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.794420:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae6de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c5a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.794429:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.794431:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845041.794434:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845041.794495:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641738112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 340us (640us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845041.794502:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1433 00000100:00100000:3.0:1788845041.797727:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.797777:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.797861:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641738240 00000100:00100000:0.0:1788845041.797871:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641738240 00000100:00100000:0.0:1788845041.797949:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1434 00000100:00100000:0.0:1788845041.797959:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641738240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.797979:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.798000:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae6e5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c61 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.798038:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.798083:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660365568:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.798299:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.798848:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641738368 00000100:00100000:2.0:1788845041.798856:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641738368 00000100:00100000:2.0:1788845041.798923:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1034 00000100:00100000:2.0:1788845041.798931:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034509450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641738368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.798943:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.798956:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae6ec lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c68 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.799000:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660365696:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.799048:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.803763:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660365696:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.803804:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae6ec lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c68 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.803829:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae6ec lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c68 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.803843:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845041.803938:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034509450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641738368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5006us (6158us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.803949:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1034 00000100:00100000:0.0:1788845041.804407:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660365568:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845041.804423:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358029 00010000:00010000:0.0:1788845041.804436:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae6e5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c61 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.804450:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.804453:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845041.804458:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae6e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c61 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.804473:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006200, rc 0) 00000100:00100000:0.0:1788845041.804513:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641738240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6555us (6795us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.804522:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1434 00000100:00100000:3.0:1788845041.809655:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.809709:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.809787:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641738624 00000100:00100000:0.0:1788845041.809798:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641738624 00000100:00100000:0.0:1788845041.809857:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1035 00000100:00100000:0.0:1788845041.809867:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641738624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.809928:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.809948:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287ae6f3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c76 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.810007:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660365824:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.810084:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.810364:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641738496 00000100:00100000:2.0:1788845041.810370:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641738496 00000100:00100000:2.0:1788845041.810413:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1435 00000100:00100000:2.0:1788845041.810421:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641738496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.810434:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.810448:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae6fa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c6f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.810477:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 23 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.810501:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008919500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660365952:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.810549:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.814965:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008919500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660365952:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845041.814994:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358029 00010000:00010000:2.0:1788845041.815013:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae6fa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c6f expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.815045:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.815049:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.815054:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae6fa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c6f expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.815068:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845041.815123:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641738496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4701us (5476us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.815131:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1435 00000100:00100000:0.0:1788845041.816988:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660365824:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.817025:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287ae6f3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c76 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.817047:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287ae6f3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c76 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.817058:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845041.817157:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641738624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7291us (7447us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.817167:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1035 00000100:00100000:3.0:1788845041.822672:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.822763:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641738752 00000100:00100000:2.0:1788845041.822774:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641738752 00002000:00100000:2.0:1788845041.822827:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91f80 x1875739641738752/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 2168/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.822844:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91f80 with x1875739641738752 00000100:00100000:2.0:1788845041.822905:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 944 00000100:00100000:2.0:1788845041.822916:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:2368:x1875739641738752:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845041.823514:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 358029 -> 358029 00000004:00010000:2.0:1788845041.823520:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:2.0:1788845041.823523:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:2.0:1788845041.823524:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:2.0:1788845041.823527:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:2.0:1788845041.823633:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2368:x1875739641738752:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 718us (970us total) trans 4294967812 rc 0/0 00000100:00100000:2.0:1788845041.823642:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 944 00002000:00100000:2.0:1788845041.823650:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91f80 x1875739641738752/t4294967812(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:147/0 lens 2168/448 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845041.823658:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91f80 with x1875739641738752 00000100:00100000:2.0:1788845041.826806:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.826913:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641738880 00000100:00100000:0.0:1788845041.826924:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641738880 00000100:00100000:0.0:1788845041.826992:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 945 00000100:00100000:0.0:1788845041.827004:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:2367:x1875739641738880:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845041.827385:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 358029 -> 358029 00000004:00010000:0.0:1788845041.827389:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.827391:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.827393:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.827395:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:0.0:1788845041.827469:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2367:x1875739641738880:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 467us (671us total) trans 4294967813 rc 0/0 00000100:00100000:0.0:1788845041.827477:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 945 00000100:00100000:1.0:1788845041.831094:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.831182:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641739008 00000100:00100000:3.0:1788845041.831192:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641739008 00000100:00100000:3.0:1788845041.831312:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1436 00000100:00100000:3.0:1788845041.831322:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641739008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845041.831347:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845041.831368:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae701 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c7d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.831454:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.831469:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.831482:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845041.831507:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 358029 00000004:00200000:3.0:1788845041.831511:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845041.831529:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.831555:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae701 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 29 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.831587:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 28 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.831601:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.831604:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845041.831607:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:3.0:1788845041.831668:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641739008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 348us (583us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845041.831675:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1436 00000100:00100000:3.0:1788845041.836020:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.836198:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641739264 00000100:00100000:1.0:1788845041.836208:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641739264 00000100:00100000:1.0:1788845041.836281:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1437 00000100:00100000:1.0:1788845041.836292:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641739264:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845041.836347:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845041.836373:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14a00/0xb2187301287ae6de lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c5a expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.836394:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.836399:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845041.836629:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.836644:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.836653:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.836687:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.836714:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 27 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.836731:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660366080:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845041.836819:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.844232:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660366080:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845041.844273:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.845616:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.845740:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641739392 00000100:00100000:0.0:1788845041.845750:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641739392 00010000:00010000:0.0:1788845041.845794:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 28 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.845843:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 494 00000100:00100000:0.0:1788845041.845852:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641739392:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845041.845865:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845041.845872:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 28 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.845922:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 28 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.845943:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287ae708 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c7d expref: 28 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.845960:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.845966:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.845976:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.845983:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.846007:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845041.846067:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641739392:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 214us (456us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845041.846075:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 494 00010000:00010000:2.0:1788845041.846397:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.846407:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845041.846527:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967814 00000004:00010000:2.0:1788845041.846579:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 358029 -> 358029 00000004:00010000:2.0:1788845041.846583:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:2.0:1788845041.846587:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:2.0:1788845041.846590:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:2.0:1788845041.846593:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:2.0:1788845041.846602:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d49c86a7 reply state 00000000c6558176 transno 4294967815 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.846612:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845041.846732:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641739264:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 10442us (10722us total) trans 4294967815 rc 0/0 00000100:00100000:2.0:1788845041.846741:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1437 00000020:00080000:2.0:1788845041.846746:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788845041.850985:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.851094:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641739520 00000100:00100000:0.0:1788845041.851103:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641739520 00000100:00100000:0.0:1788845041.851157:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1438 00000100:00100000:0.0:1788845041.851167:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641739520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.851191:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.851219:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c84 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.851249:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.851274:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.851308:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.851325:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 26 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.851339:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660366208:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845041.851480:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.854627:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660366208:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845041.854665:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.854685:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.854688:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.854701:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.854711:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006200, rc 0) 00000100:00100000:3.0:1788845041.854770:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641739520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3603us (3792us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845041.854778:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1438 00010000:00010000:1.0:1788845041.855775:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.855788:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.855796:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.855802:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d14e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845041.855807:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.855815:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.855829:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.855832:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845041.855835:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.855841:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287ae70f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.856982:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.857095:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641739648 00000100:00100000:3.0:1788845041.857105:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641739648 00010000:00010000:3.0:1788845041.857143:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 27 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.857182:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 495 00000100:00100000:3.0:1788845041.857192:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d13450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641739648:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845041.857205:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845041.857242:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 27 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.857266:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 27 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.857288:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287ae5b1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b6c expref: 27 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.857302:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.857307:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.857317:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.857326:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.857364:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.857382:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d93800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660366336:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845041.857457:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.860673:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.860761:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641739776 00000100:00100000:1.0:1788845041.860770:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641739776 00000100:00100000:1.0:1788845041.860828:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1036 00000100:00100000:1.0:1788845041.860839:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641739776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.860859:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.860878:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c8b expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.860946:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.860955:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(12 14) 00010000:00010000:1.0:1788845041.860960:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 14) 00010000:00010000:1.0:1788845041.860965:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.861000:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.861017:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 6 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.861030:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660366464:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845041.861097:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.861663:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d93800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660366336:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845041.861683:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845041.861734:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d13450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641739648:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4543us (4759us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845041.861744:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 495 00000100:00100000:0.0:1788845041.867021:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.867143:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660366464:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845041.867180:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.867202:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14a00, rc 0) 00000100:00100000:1.0:1788845041.867268:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641739776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6430us (6604us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845041.867276:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1036 00000100:00100000:2.0:1788845041.867366:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641739904 00000100:00100000:2.0:1788845041.867374:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641739904 00010000:00010000:2.0:1788845041.867406:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 7 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.867441:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 496 00000100:00100000:2.0:1788845041.867450:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641739904:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845041.867465:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845041.867488:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 7 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.867507:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 7 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.867533:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287ae55d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1400831) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9b1f expref: 7 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.867553:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 14) 00010000:00010000:2.0:1788845041.867558:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 14) 00010000:00010000:2.0:1788845041.867577:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.867588:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.867623:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.867636:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660366592:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845041.867689:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.872608:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.872654:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.872808:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641740032 00000100:00100000:1.0:1788845041.872817:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641740032 00002000:00010000:1.0:1788845041.872856:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845041.872861:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f99500 with x1875739641740032 00000100:00100000:1.0:1788845041.872921:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 946 00000100:00100000:1.0:1788845041.872932:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641740032:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845041.873132:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 358029 -> 369456 00000004:00010000:1.0:1788845041.873136:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.873138:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.873141:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.873143:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:1.0:1788845041.873221:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641740032:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 279us (610us total) trans 4294967816 rc 0/0 00000100:00100000:1.0:1788845041.873230:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 946 00002000:00010000:1.0:1788845041.873234:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845041.873236:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f99500 with x1875739641740032 00000100:00100000:2.0:1788845041.873400:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660366592:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845041.873423:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845041.873447:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641739904:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5999us (6437us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.873456:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 496 00000100:00100000:3.0:1788845041.874011:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641740160 00000100:00100000:3.0:1788845041.874018:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641740160 00002000:00010000:3.0:1788845041.874046:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845041.874050:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383d500 with x1875739641740160 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845041.874059:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383d500. 00000100:00100000:3.0:1788845041.874082:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 485 00000100:00100000:3.0:1788845041.874090:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641740160:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845041.874279:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1788845041.874283:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.874286:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.874288:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.874290:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 163 -> 163 00000100:00100000:3.0:1788845041.874325:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641740160:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 236us (1669us total) trans 4294967480 rc 0/0 00000100:00100000:3.0:1788845041.874332:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 485 00002000:00010000:3.0:1788845041.874335:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845041.874338:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383d500 with x1875739641740160 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845041.874344:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383d500. 00000100:00100000:0.0:1788845041.877628:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.877743:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641740288 00000100:00100000:2.0:1788845041.877754:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641740288 00000100:00100000:2.0:1788845041.877815:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1439 00000100:00100000:2.0:1788845041.877826:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:14891:x1875739641740288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.877845:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.877867:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae724 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c92 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.877995:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.878011:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.878024:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845041.878053:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 369456 00000004:00200000:2.0:1788845041.878058:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845041.878075:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.878104:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae724 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 28 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.878122:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 27 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.878131:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.878134:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845041.878138:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033000, rc 0) 00000100:00100000:2.0:1788845041.878189:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641740288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 366us (572us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.878197:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1439 00000100:00100000:0.0:1788845041.880869:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.880996:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641740416 00000100:00100000:2.0:1788845041.881007:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641740416 00000100:00100000:2.0:1788845041.881066:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1440 00000100:00100000:2.0:1788845041.881077:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641740416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.881097:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.881119:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae732 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9c99 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.881153:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.881193:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660366720:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.881274:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.884313:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.884480:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660366720:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845041.884507:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 369456 00010000:00010000:2.0:1788845041.884528:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae732 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c99 expref: 28 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.884549:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.884553:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.884558:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae732 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c99 expref: 28 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.884586:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845041.884647:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641740416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3571us (3787us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.884656:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1440 00000100:00100000:1.0:1788845041.884755:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641740544 00000100:00100000:1.0:1788845041.884763:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641740544 00000100:00100000:1.0:1788845041.884816:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1037 00000100:00100000:1.0:1788845041.884826:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641740544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.884841:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.884858:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae739 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ca0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.884953:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660366848:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.885006:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.888428:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660366848:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.888469:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae739 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ca0 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.888495:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287ae739 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ca0 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.888508:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845041.888566:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641740544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3740us (4263us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.889089:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1037 00000100:00100000:0.0:1788845041.892316:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.892407:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641740672 00000100:00100000:2.0:1788845041.892417:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641740672 00000100:00100000:2.0:1788845041.892482:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1441 00000100:00100000:2.0:1788845041.892496:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641740672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.892517:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.892537:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae740 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ca7 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.892588:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.892631:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660366976:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.892719:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.895086:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.895178:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641740800 00000100:00100000:2.0:1788845041.895187:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641740800 00000100:00100000:2.0:1788845041.895252:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1038 00000100:00100000:2.0:1788845041.895271:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641740800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.895290:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.895308:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae747 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cae expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.895369:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660367104:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.895449:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.897841:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660366976:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845041.897869:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 369456 00010000:00010000:2.0:1788845041.897931:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae740 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ca7 expref: 28 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.897960:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.897964:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845041.897971:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae740 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ca7 expref: 28 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.897987:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845041.898354:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641740672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5861us (6047us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.898364:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1441 00000100:00100000:2.0:1788845041.900611:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660367104:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845041.900652:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae747 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cae expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.900677:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae747 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cae expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.900702:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:2.0:1788845041.900749:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641740800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5480us (5672us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845041.900758:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1038 00000100:00100000:2.0:1788845041.907716:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.907810:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641741056 00000100:00100000:3.0:1788845041.907821:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641741056 00000100:00100000:3.0:1788845041.907918:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1442 00000100:00100000:3.0:1788845041.907929:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641741056:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:0.0:1788845041.908556:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.908653:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.908668:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.908716:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.908736:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 26 pid: 5841 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.908751:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660367232:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845041.908871:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d60c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845041.911678:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660367232:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845041.911717:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845041.913372:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.913687:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641741184 00000100:00100000:2.0:1788845041.913707:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641741184 00010000:00010000:2.0:1788845041.913746:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 27 pid: 5841 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.913787:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 497 00000100:00100000:2.0:1788845041.913797:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641741184:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845041.913811:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845041.913818:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 27 pid: 5841 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.913839:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 27 pid: 5841 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.913861:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae72b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c92 expref: 27 pid: 5841 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.913877:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.913922:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.913934:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.913942:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.916496:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845041.916594:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641741184:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2792us (3224us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.916608:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 497 00010000:00010000:0.0:1788845041.918054:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.918068:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845041.918189:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967817 00000004:00010000:0.0:1788845041.918255:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 369456 -> 369456 00000004:00010000:0.0:1788845041.918259:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.918262:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.918264:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845041.918266:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:0.0:1788845041.918273:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000095e7336 reply state 00000000fef7a6c7 transno 4294967818 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.918283:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845041.918466:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641741056:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 10536us (10757us total) trans 4294967818 rc 0/0 00000100:00100000:0.0:1788845041.918474:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1442 00000020:00080000:0.0:1788845041.918479:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845041.922109:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.922194:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845041.922906:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641741440 00000100:00100000:3.0:1788845041.922918:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641741440 00002000:00010000:3.0:1788845041.922963:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845041.922969:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383f480 with x1875739641741440 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845041.922982:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383f480. 00000100:00100000:3.0:1788845041.923016:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 486 00000100:00100000:3.0:1788845041.923030:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641741440:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845041.923335:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1788845041.923339:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.923342:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.923345:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00002000:00010000:3.0:1788845041.923347:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 163 -> 163 00010000:00010000:2.0:1788845041.923491:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.923503:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.923511:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.923522:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845041.923528:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.923537:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.923553:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.923556:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1788845041.923557:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641741312 00000100:00100000:1.0:1788845041.923562:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641741312 00010000:00010000:2.0:1788845041.923571:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.923578:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae74e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845041.923583:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845041.923587:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c700 with x1875739641741312 00000100:00100000:1.0:1788845041.923611:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 947 00000100:00100000:1.0:1788845041.923618:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641741312:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845041.923771:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 369456 -> 640670 00000004:00010000:1.0:1788845041.923775:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.923778:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.923780:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845041.923782:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:3.0:1788845041.924093:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641741440:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 1065us (1896us total) trans 4294967481 rc 0/0 00000100:00100000:3.0:1788845041.924102:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 486 00002000:00010000:3.0:1788845041.924107:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845041.924110:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383f480 with x1875739641741440 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845041.924116:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383f480. 00000100:00100000:1.0:1788845041.926061:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641741312:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 2433us (3951us total) trans 4294967819 rc 0/0 00000100:00100000:1.0:1788845041.926074:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 947 00002000:00010000:1.0:1788845041.926080:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845041.926084:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383c700 with x1875739641741312 00000100:00100000:2.0:1788845041.929616:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.929727:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641741568 00000100:00100000:1.0:1788845041.929738:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641741568 00000100:00100000:1.0:1788845041.929806:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1443 00000100:00100000:1.0:1788845041.929815:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:14891:x1875739641741568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.929838:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.929859:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae755 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cb5 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.929980:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.929995:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.930006:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845041.930032:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 640670 00000004:00200000:1.0:1788845041.930037:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845041.930058:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.930083:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae755 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.930100:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.930108:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.930111:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845041.930114:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:1.0:1788845041.930164:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641741568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 350us (555us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845041.930171:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1443 00000100:00100000:0.0:1788845041.933072:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.933117:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.933221:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641741696 00000100:00100000:2.0:1788845041.933231:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641741696 00000100:00100000:2.0:1788845041.933287:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1444 00000100:00100000:2.0:1788845041.933297:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641741696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.933318:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.933338:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae763 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cbc expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.933378:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845041.933420:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660367360:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845041.933501:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.934045:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641741824 00000100:00100000:1.0:1788845041.934051:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641741824 00000100:00100000:1.0:1788845041.934097:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1039 00000100:00100000:1.0:1788845041.934105:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641741824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.934116:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.934129:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae76a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cc3 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.934170:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660367488:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.934226:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845041.936411:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660367360:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845041.936437:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:1.0:1788845041.936457:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae763 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cbc expref: 28 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.936481:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845041.936484:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845041.936490:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033000/0xb2187301287ae763 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cbc expref: 28 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.936507:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033000, rc 0) 00000100:00100000:1.0:1788845041.936576:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641741696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3272us (3505us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.936585:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1444 00000100:00100000:1.0:1788845041.940086:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660367488:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.940127:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae76a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cc3 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.940152:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae76a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cc3 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.940166:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845041.940238:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641741824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6133us (7119us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.940247:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1039 00000100:00100000:3.0:1788845041.943438:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.943680:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641741952 00000100:00100000:1.0:1788845041.943698:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641741952 00000100:00100000:1.0:1788845041.943760:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1445 00000100:00100000:1.0:1788845041.943770:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641741952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.943794:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.943815:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae771 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cca expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.943852:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.943924:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660367616:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.944009:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845041.946450:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.946606:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641742080 00000100:00100000:1.0:1788845041.946616:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641742080 00000100:00100000:1.0:1788845041.946673:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1040 00000100:00100000:1.0:1788845041.946684:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641742080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845041.946731:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845041.946749:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae778 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cd1 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845041.946809:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660367744:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845041.946915:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.947223:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660367616:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845041.947244:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:3.0:1788845041.947260:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae771 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cca expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.947277:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845041.947280:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845041.947285:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287ae771 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cca expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845041.947299:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:3.0:1788845041.947345:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641741952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3576us (3917us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845041.947352:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1445 00000100:00100000:1.0:1788845041.949947:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660367744:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845041.949987:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae778 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cd1 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.950011:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae778 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cd1 expref: 7 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845041.950028:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845041.950075:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641742080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3392us (3632us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845041.950082:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1040 00000100:00100000:0.0:1788845041.955475:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845041.955557:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641742208 00000100:00100000:1.0:1788845041.955576:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641742208 00002000:00100000:1.0:1788845041.955624:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383e680 x1875739641742208/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/0 e 0 to 0 dl 1788845052 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845041.955636:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383e680 with x1875739641742208 00000100:00100000:1.0:1788845041.955673:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 948 00000100:00100000:1.0:1788845041.955683:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641742208:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845041.982189:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641742208:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 26497us (26713us total) trans 0 rc 643072/643072 00000100:00100000:1.0:1788845041.982203:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 948 00002000:00100000:1.0:1788845041.982216:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383e680 x1875739641742208/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:147/0 lens 488/440 e 0 to 0 dl 1788845052 ref 1 fl Complete:/600/0 rc 643072/643072 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845041.982227:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383e680 with x1875739641742208 00000100:00100000:0.0:1788845041.985527:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.985676:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641742336 00000100:00100000:2.0:1788845041.985696:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641742336 00000100:00100000:2.0:1788845041.985758:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1446 00000100:00100000:2.0:1788845041.985769:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641742336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845041.985791:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845041.985814:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae77f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cd8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.985945:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae786 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.985962:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae786 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.985975:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae786 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845041.986002:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 640670 00000004:00200000:2.0:1788845041.986006:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845041.986024:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae786 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.986053:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae77f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cd8 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.986071:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae786 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cd8 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845041.986080:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845041.986083:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845041.986086:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:2.0:1788845041.986138:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641742336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 371us (620us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845041.986145:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1446 00000100:00100000:2.0:1788845041.990608:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845041.990810:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845041.990916:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641742464 00000100:00100000:0.0:1788845041.990928:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641742464 00000100:00100000:0.0:1788845041.990994:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1447 00000100:00100000:0.0:1788845041.991005:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641742464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.991029:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.991051:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae78d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cdf expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.991089:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.991131:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660367872:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.991230:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d60c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845041.993084:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641742592 00000100:00100000:3.0:1788845041.993099:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641742592 00000100:00100000:0.0:1788845041.993461:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1041 00000100:00100000:0.0:1788845041.993472:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641742592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845041.993489:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845041.993506:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287ae794 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ce6 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845041.993558:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660368000:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845041.993921:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000004:00010000:0.0:1788845041.995216:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845041 -> 1788845042 00000100:00100000:0.0:1788845041.995248:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660367872:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845041.995271:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:0.0:1788845041.995290:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae78d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cdf expref: 28 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.995310:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845041.995313:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845041.995317:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae78d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cdf expref: 28 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.995333:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845041.995381:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641742464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4378us (4571us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.995388:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1447 00002000:00010000:0.0:1788845041.998842:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845041 -> 1788845042 00000100:00100000:0.0:1788845041.998873:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660368000:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845041.998926:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287ae794 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ce6 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.998955:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287ae794 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ce6 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845041.998969:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:0.0:1788845041.999118:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641742592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5646us (8520us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845041.999127:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1041 00000100:00100000:1.0:1788845042.005758:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.005860:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641742720 00000100:00100000:2.0:1788845042.005870:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641742720 00000100:00100000:0.0:1788845042.006184:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1448 00000100:00100000:0.0:1788845042.006196:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641742720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.006226:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.006247:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae79b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ced expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.006417:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.006428:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae79b lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ced expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.006464:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.006482:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 16 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.006503:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660368128:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845042.006660:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.010607:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.010738:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641742848 00000100:00100000:1.0:1788845042.010748:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641742848 00010000:00010000:1.0:1788845042.010790:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 17 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.010830:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 498 00000100:00100000:1.0:1788845042.010840:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641742848:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845042.010854:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845042.010878:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 17 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.010923:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 17 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.010947:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845042.010953:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae79b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ced expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.010964:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae79b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ced expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.010973:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae79b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ced expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.011013:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae79b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ced expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.011027:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660368256:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845042.011104:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00010000:00010000:0.0:1788845042.013658:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae716 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c84 expref: 17 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.013683:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660368128:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845042.013724:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae79b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ced expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.013740:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.013742:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.013746:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845042.013869:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641742720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7675us (8122us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.013878:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1448 00000100:00100000:1.0:1788845042.014816:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660368256:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845042.014835:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845042.014880:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641742848:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4040us (4283us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845042.014907:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 498 00000100:00100000:2.0:1788845042.016874:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.017911:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641742976 00000100:00100000:1.0:1788845042.017921:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641742976 00002000:00100000:1.0:1788845042.017972:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a00891b100 x1875739641742976/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.017983:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a00891b100 with x1875739641742976 00000100:00100000:1.0:1788845042.018029:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 949 00000100:00100000:1.0:1788845042.018039:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641742976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.018300:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641742976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (1433us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845042.018308:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 949 00002000:00100000:1.0:1788845042.018315:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a00891b100 x1875739641742976/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.018323:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a00891b100 with x1875739641742976 00000100:00100000:3.0:1788845042.020913:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.021200:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641743104 00000100:00100000:2.0:1788845042.021224:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641743104 00002000:00100000:2.0:1788845042.021273:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91180 x1875739641743104/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.021285:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91180 with x1875739641743104 00000100:00100000:2.0:1788845042.021325:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 950 00000100:00100000:2.0:1788845042.021335:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641743104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.021533:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641743104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 199us (650us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845042.021541:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 950 00002000:00100000:2.0:1788845042.021547:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91180 x1875739641743104/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.021554:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91180 with x1875739641743104 00000100:00100000:1.0:1788845042.025592:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.025668:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641743232 00000100:00100000:0.0:1788845042.025681:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641743232 00002000:00100000:0.0:1788845042.025741:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9500 x1875739641743232/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.025753:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9500 with x1875739641743232 00000100:00100000:0.0:1788845042.025785:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 951 00000100:00100000:0.0:1788845042.025796:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641743232:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.026052:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641743232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 257us (469us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.026063:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 951 00002000:00100000:0.0:1788845042.026070:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9500 x1875739641743232/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.026078:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9500 with x1875739641743232 00000100:00100000:0.0:1788845042.029835:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.030083:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641743360 00000100:00100000:1.0:1788845042.030093:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641743360 00002000:00100000:1.0:1788845042.030143:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383dc00 x1875739641743360/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.030154:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383dc00 with x1875739641743360 00000100:00100000:1.0:1788845042.030195:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 952 00000100:00100000:1.0:1788845042.030205:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641743360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.030619:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641743360:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 414us (793us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845042.030631:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 952 00002000:00100000:1.0:1788845042.030639:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383dc00 x1875739641743360/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.030648:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383dc00 with x1875739641743360 00000100:00100000:1.0:1788845042.035465:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.035743:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641743488 00000100:00100000:0.0:1788845042.035755:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641743488 00002000:00100000:0.0:1788845042.035819:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359bb800 x1875739641743488/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.035833:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359bb800 with x1875739641743488 00000100:00100000:0.0:1788845042.036017:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 953 00000100:00100000:0.0:1788845042.036036:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641743488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.036508:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641743488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 478us (1054us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.036517:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 953 00002000:00100000:0.0:1788845042.036525:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359bb800 x1875739641743488/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.036533:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359bb800 with x1875739641743488 00000100:00100000:2.0:1788845042.044309:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.044496:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641743616 00000100:00100000:1.0:1788845042.044507:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641743616 00002000:00100000:1.0:1788845042.044557:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918000 x1875739641743616/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.044577:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918000 with x1875739641743616 00000100:00100000:1.0:1788845042.044614:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 954 00000100:00100000:1.0:1788845042.044625:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641743616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.044857:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641743616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 232us (556us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845042.044864:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 954 00002000:00100000:1.0:1788845042.044871:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918000 x1875739641743616/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.044879:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918000 with x1875739641743616 00000100:00100000:2.0:1788845042.048583:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.048829:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641743744 00000100:00100000:3.0:1788845042.048839:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641743744 00002000:00100000:3.0:1788845042.048925:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0000 x1875739641743744/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.048936:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0000 with x1875739641743744 00000100:00100000:3.0:1788845042.048964:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 955 00000100:00100000:3.0:1788845042.048974:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641743744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.049365:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641743744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 390us (802us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.049375:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 955 00002000:00100000:3.0:1788845042.049382:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0000 x1875739641743744/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.049390:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0000 with x1875739641743744 00000100:00100000:3.0:1788845042.053297:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.053577:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641743872 00000100:00100000:1.0:1788845042.053588:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641743872 00002000:00100000:1.0:1788845042.053638:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91500 x1875739641743872/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.053649:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91500 with x1875739641743872 00000100:00100000:1.0:1788845042.053714:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 956 00000100:00100000:1.0:1788845042.053727:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641743872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.053974:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641743872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (686us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845042.053981:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 956 00002000:00100000:1.0:1788845042.053988:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d91500 x1875739641743872/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.053995:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d91500 with x1875739641743872 00000100:00100000:2.0:1788845042.057720:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.057810:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641744000 00000100:00100000:3.0:1788845042.057820:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641744000 00002000:00100000:3.0:1788845042.057869:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0380 x1875739641744000/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.057918:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0380 with x1875739641744000 00000100:00100000:3.0:1788845042.057950:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 957 00000100:00100000:3.0:1788845042.057959:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641744000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.058187:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641744000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 228us (476us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.058193:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 957 00002000:00100000:3.0:1788845042.058200:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb0380 x1875739641744000/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.058345:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb0380 with x1875739641744000 00000100:00100000:0.0:1788845042.061947:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.062032:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641744128 00000100:00100000:1.0:1788845042.062043:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641744128 00002000:00100000:1.0:1788845042.062093:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383ce00 x1875739641744128/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.062104:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383ce00 with x1875739641744128 00000100:00100000:1.0:1788845042.062144:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 958 00000100:00100000:1.0:1788845042.062154:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641744128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.062387:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641744128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (448us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845042.062395:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 958 00002000:00100000:1.0:1788845042.062402:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0383ce00 x1875739641744128/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.062410:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0383ce00 with x1875739641744128 00000100:00100000:1.0:1788845042.066759:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.066850:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641744256 00000100:00100000:0.0:1788845042.066860:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641744256 00002000:00100000:0.0:1788845042.066936:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359bb100 x1875739641744256/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.066948:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359bb100 with x1875739641744256 00000100:00100000:0.0:1788845042.066993:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 959 00000100:00100000:0.0:1788845042.067002:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641744256:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.067457:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641744256:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 454us (707us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.067466:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 959 00002000:00100000:0.0:1788845042.067474:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359bb100 x1875739641744256/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.067482:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359bb100 with x1875739641744256 00000100:00100000:3.0:1788845042.070662:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.070787:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641744384 00000100:00100000:1.0:1788845042.070798:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641744384 00002000:00100000:1.0:1788845042.070848:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90000 x1875739641744384/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.070863:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90000 with x1875739641744384 00000100:00100000:1.0:1788845042.070933:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 960 00000100:00100000:1.0:1788845042.070942:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641744384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.071183:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641744384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 241us (531us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845042.071191:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 960 00002000:00100000:1.0:1788845042.071198:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90000 x1875739641744384/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.071217:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90000 with x1875739641744384 00000100:00100000:2.0:1788845042.075318:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.075464:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641744512 00000100:00100000:3.0:1788845042.075475:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641744512 00002000:00100000:3.0:1788845042.075526:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1180 x1875739641744512/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.075537:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1180 with x1875739641744512 00000100:00100000:3.0:1788845042.075588:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 961 00000100:00100000:3.0:1788845042.075599:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641744512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.075840:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641744512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 242us (531us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.075848:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 961 00002000:00100000:3.0:1788845042.075855:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01bb1180 x1875739641744512/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.075862:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01bb1180 with x1875739641744512 00000100:00100000:3.0:1788845042.079578:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.079701:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641744640 00000100:00100000:1.0:1788845042.079711:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641744640 00000100:00100000:1.0:1788845042.079781:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1449 00000100:00100000:1.0:1788845042.079793:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641744640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845042.079816:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845042.079838:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cf4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.079877:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845042.079928:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:1.0:1788845042.079937:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.079950:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845042.079954:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845042.079958:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14400, rc 0) 00000100:00100000:1.0:1788845042.080011:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641744640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 221us (454us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845042.080017:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1449 00000100:00100000:2.0:1788845042.084297:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.084462:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641744768 00000100:00100000:1.0:1788845042.084471:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641744768 00000100:00100000:1.0:1788845042.084531:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1450 00000100:00100000:1.0:1788845042.084541:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641744768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845042.084562:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845042.084576:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788845042.084626:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287ae79b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ced expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.084647:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845042.084652:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845042.084665:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9cfb expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.084699:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.084709:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 28 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.084744:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.084767:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 16 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.084781:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660368384:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845042.084876:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.089482:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.089591:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660368384:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845042.089633:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 28 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.089657:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845042.089661:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845042.089667:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 28 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.089678:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15a00, rc 0) 00000100:00100000:1.0:1788845042.089751:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641744768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5211us (5465us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845042.089760:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1450 00000100:00100000:3.0:1788845042.090329:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641744896 00000100:00100000:3.0:1788845042.090338:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641744896 00010000:00010000:3.0:1788845042.090370:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 17 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.090405:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 499 00000100:00100000:3.0:1788845042.090414:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641744896:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845042.090427:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845042.090452:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 17 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.090468:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 17 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.090489:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14400/0xb2187301287ae7a2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cf4 expref: 17 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.090502:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.090506:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.090520:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.090529:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.090574:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.090590:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660368512:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845042.090659:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.093839:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660368512:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845042.093865:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845042.094065:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641744896:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3649us (4592us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.094078:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 499 00000100:00100000:1.0:1788845042.097972:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.098117:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641745024 00000100:00100000:0.0:1788845042.098129:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641745024 00002000:00100000:0.0:1788845042.098187:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9180 x1875739641745024/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.098199:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9180 with x1875739641745024 00000100:00100000:0.0:1788845042.098252:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 962 00000100:00100000:0.0:1788845042.098271:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2368:x1875739641745024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.098579:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2368:x1875739641745024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 308us (616us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.098586:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 962 00002000:00100000:0.0:1788845042.098592:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b9180 x1875739641745024/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.098600:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b9180 with x1875739641745024 00000100:00100000:3.0:1788845042.102016:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.102134:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641745152 00000100:00100000:1.0:1788845042.102145:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641745152 00002000:00100000:1.0:1788845042.102203:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90a80 x1875739641745152/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.102227:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90a80 with x1875739641745152 00000100:00100000:1.0:1788845042.102280:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 963 00000100:00100000:1.0:1788845042.102291:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641745152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.102574:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641745152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 284us (568us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845042.102583:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 963 00002000:00100000:1.0:1788845042.102590:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90a80 x1875739641745152/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.102598:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90a80 with x1875739641745152 00000100:00100000:1.0:1788845042.107036:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.107169:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641745280 00000100:00100000:0.0:1788845042.107180:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641745280 00000100:00100000:0.0:1788845042.107251:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1451 00000100:00100000:0.0:1788845042.107271:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641745280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.107298:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.107322:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae7b0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d02 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.107365:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.107406:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660368640:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.107494:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00000004:00010000:0.0:1788845042.110199:0:11036:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845041 -> 1788845042 00000004:00010000:0.0:1788845042.110224:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845041 -> 1788845042 00000100:00100000:0.0:1788845042.110367:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660368640:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.110405:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:0.0:1788845042.110428:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae7b0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d02 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.110450:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.110453:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.110458:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae7b0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d02 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.110474:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:0.0:1788845042.110591:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641745280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3321us (3564us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.110600:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1451 00000100:00100000:1.0:1788845042.114489:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.114588:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641745408 00000100:00100000:0.0:1788845042.114598:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641745408 00000100:00100000:0.0:1788845042.114662:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1452 00000100:00100000:0.0:1788845042.114672:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641745408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.114705:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.114726:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae7b7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d09 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.114765:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.114806:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660368768:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.114937:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.118659:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660368768:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.118696:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:0.0:1788845042.118718:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae7b7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d09 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.118739:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.118742:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.118747:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae7b7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d09 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.118762:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845042.118827:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641745408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4157us (4348us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.118835:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1452 00000100:00100000:3.0:1788845042.125425:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.125816:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641745536 00000100:00100000:0.0:1788845042.125826:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641745536 00002000:00100000:0.0:1788845042.125876:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410380 x1875739641745536/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 504/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.125922:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410380 with x1875739641745536 00000100:00100000:0.0:1788845042.125960:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 964 00000100:00100000:0.0:1788845042.125970:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641745536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.152714:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2367:x1875739641745536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 26734us (27289us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1788845042.152729:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 964 00002000:00100000:0.0:1788845042.152742:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03410380 x1875739641745536/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 504/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 614400/614400 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.152753:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03410380 with x1875739641745536 00000100:00100000:1.0:1788845042.156437:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.156528:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641745664 00000100:00100000:2.0:1788845042.156537:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641745664 00000100:00100000:2.0:1788845042.156595:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1042 00000100:00100000:2.0:1788845042.156604:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641745664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.156623:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.156639:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ae7be lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d10 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.156719:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660368896:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.156797:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845042.159616:0:6682:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845041 -> 1788845042 00002000:00010000:1.0:1788845042.159626:0:6682:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845041 -> 1788845042 00000100:00100000:1.0:1788845042.159651:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660368896:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845042.159695:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ae7be lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d10 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.159721:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ae7be lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d10 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.159735:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033800, rc 0) 00000100:00100000:1.0:1788845042.159798:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641745664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3194us (3370us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845042.159808:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1042 00000100:00100000:3.0:1788845042.163112:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.163229:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641745792 00000100:00100000:2.0:1788845042.163241:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641745792 00000100:00100000:2.0:1788845042.163313:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1043 00000100:00100000:2.0:1788845042.163324:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641745792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.163343:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.163362:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000028aee44e/0xb2187301287ae7c5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d17 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.163421:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660369024:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.163508:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.166644:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660369024:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845042.166700:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000028aee44e/0xb2187301287ae7c5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d17 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.166728:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028aee44e/0xb2187301287ae7c5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d17 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.166744:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033000, rc 0) 00000100:00100000:1.0:1788845042.166853:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641745792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3529us (3749us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845042.166862:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1043 00000100:00100000:1.0:1788845042.171649:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.171822:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641745920 00000100:00100000:0.0:1788845042.171833:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641745920 00000100:00100000:0.0:1788845042.172013:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1044 00000100:00100000:0.0:1788845042.172026:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641745920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.172048:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.172066:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d1e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.172104:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.172115:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 14) 00010000:00010000:0.0:1788845042.172120:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(14 14) 00010000:00010000:0.0:1788845042.172127:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1740800->1748991] (req 1740800->1748991) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.172165:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.172186:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 7 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.172202:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660369152:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845042.172353:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.175784:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.175872:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641746048 00000100:00100000:3.0:1788845042.175915:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641746048 00010000:00010000:3.0:1788845042.175957:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 8 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.175992:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 500 00000100:00100000:3.0:1788845042.176001:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641746048:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845042.176020:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845042.176044:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 8 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.176062:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 8 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.176091:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(14 14) 00010000:00010000:3.0:1788845042.176096:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(14 14) 00010000:00010000:3.0:1788845042.176105:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.176116:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.176154:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.176170:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f776300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660369280:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845042.176273:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.178788:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f776300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660369280:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845042.178817:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845042.178872:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641746048:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2870us (3096us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.178922:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 500 00010000:00010000:0.0:1788845042.180534:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edd26e43/0xb2187301287ae71d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9c8b expref: 7 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.180558:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660369152:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845042.180650:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.180667:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845042.180791:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641745920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8767us (9152us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.180800:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1044 00000100:00100000:2.0:1788845042.184771:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.184868:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641746176 00000100:00100000:1.0:1788845042.184878:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641746176 00000100:00100000:1.0:1788845042.184973:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1453 00000100:00100000:1.0:1788845042.184983:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641746176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845042.185004:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845042.185024:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae7d3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d25 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.185061:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.185107:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660369408:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845042.185180:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.187977:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.188317:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641746304 00000100:00100000:0.0:1788845042.188327:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641746304 00000100:00100000:0.0:1788845042.188379:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1045 00000100:00100000:0.0:1788845042.188389:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641746304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.188408:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.188424:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287ae7da lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d2c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.188483:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660369536:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.188581:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.189179:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660369408:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845042.189201:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:1.0:1788845042.189227:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae7d3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d25 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.189246:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845042.189249:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845042.189255:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae7d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d25 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.189268:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:1.0:1788845042.189305:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641746176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4324us (4543us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845042.189313:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1453 00002000:00010000:0.0:1788845042.193484:0:6684:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1748158 00000100:00100000:0.0:1788845042.193518:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660369536:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845042.193556:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287ae7da lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d2c expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.193594:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287ae7da lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d2c expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.193609:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:0.0:1788845042.193677:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641746304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5288us (5707us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.193694:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1045 00000100:00100000:3.0:1788845042.197793:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.197839:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.198061:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641746560 00000100:00100000:2.0:1788845042.198075:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641746560 00000100:00100000:2.0:1788845042.198133:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1046 00000100:00100000:2.0:1788845042.198144:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641746560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.198163:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.198182:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae7e1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d3a expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.198249:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660369664:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.198349:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.198779:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641746432 00000100:00100000:1.0:1788845042.198785:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641746432 00000100:00100000:1.0:1788845042.198826:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1454 00000100:00100000:1.0:1788845042.198834:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641746432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845042.198845:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845042.198859:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae7e8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d33 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.198926:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.198955:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660369792:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845042.198996:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.203125:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660369792:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.203156:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:0.0:1788845042.203177:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae7e8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d33 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.203201:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.203218:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.203226:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15600/0xb2187301287ae7e8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d33 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.203244:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15600, rc 0) 00000100:00100000:0.0:1788845042.203309:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641746432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4475us (5525us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.203317:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1454 00000100:00100000:1.0:1788845042.203759:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660369664:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845042.203784:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae7e1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d3a expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.203801:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009a0e9eb7/0xb2187301287ae7e1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d3a expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.203812:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:1.0:1788845042.203840:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641746560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5698us (6000us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845042.203846:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1046 00000100:00100000:3.0:1788845042.207957:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.207994:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.208093:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641746688 00000100:00100000:0.0:1788845042.208103:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641746688 00000100:00100000:0.0:1788845042.208166:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1455 00000100:00100000:0.0:1788845042.208176:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641746688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.208199:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.208233:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae7ef lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d41 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.208278:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.208320:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660369920:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.208397:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.208869:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641746816 00000100:00100000:2.0:1788845042.208876:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641746816 00000100:00100000:2.0:1788845042.208944:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1047 00000100:00100000:2.0:1788845042.208953:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641746816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.208963:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.208975:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae7f6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d48 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.209015:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660370048:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.209047:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.214312:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660370048:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.214354:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae7f6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d48 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.214383:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085ae6bbe/0xb2187301287ae7f6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d48 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.214397:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:2.0:1788845042.214462:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641746816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5509us (6466us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.214471:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1047 00000100:00100000:0.0:1788845042.215840:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660369920:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.215860:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:0.0:1788845042.215876:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae7ef lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d41 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.215940:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.215944:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.215950:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae7ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d41 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.215965:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:0.0:1788845042.216025:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641746688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7849us (8076us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.216033:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1455 00000100:00100000:3.0:1788845042.219238:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.219288:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.219425:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641747072 00000100:00100000:2.0:1788845042.219435:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641747072 00000100:00100000:2.0:1788845042.219494:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1048 00000100:00100000:2.0:1788845042.219504:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641747072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.219522:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.219540:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ae7fd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d56 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.219611:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660370176:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.219706:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.219992:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641746944 00000100:00100000:0.0:1788845042.219997:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641746944 00000100:00100000:0.0:1788845042.220035:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1456 00000100:00100000:0.0:1788845042.220043:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641746944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.220055:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.220068:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae804 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d4f expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.220097:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.220121:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660370304:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.220164:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.223981:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660370304:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.224008:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:0.0:1788845042.224027:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae804 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d4f expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.224050:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.224053:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.224059:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae804 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d4f expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.224076:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845042.224133:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641746944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4090us (4903us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.224143:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1456 00000100:00100000:2.0:1788845042.227947:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660370176:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.227992:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ae7fd lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d56 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.228017:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ae7fd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d56 expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.228031:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:2.0:1788845042.228095:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641747072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8593us (8806us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.228105:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1048 00000100:00100000:2.0:1788845042.232095:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.232223:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641747200 00000100:00100000:3.0:1788845042.232233:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641747200 00000100:00100000:3.0:1788845042.232305:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1457 00000100:00100000:3.0:1788845042.232315:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641747200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.232339:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.232359:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae80b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d5d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.232399:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.232439:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f775180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660370432:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.232584:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.234723:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.234816:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641747328 00000100:00100000:0.0:1788845042.234825:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641747328 00000100:00100000:0.0:1788845042.234880:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1049 00000100:00100000:0.0:1788845042.234923:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641747328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.234944:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.234962:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae812 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d64 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.235013:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660370560:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.235095:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.238617:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660370560:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845042.238659:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae812 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d64 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.238710:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287ae812 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d64 expref: 8 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.238725:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:3.0:1788845042.238929:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f775180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660370432:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.238946:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:3.0:1788845042.238959:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae80b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d5d expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.238973:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.238976:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.238980:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae80b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d5d expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.238997:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:0.0:1788845042.239189:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641747328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4266us (4472us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.239199:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1049 00000100:00100000:3.0:1788845042.239572:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641747200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7256us (7483us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.239580:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1457 00000100:00100000:3.0:1788845042.243002:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.243035:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.243104:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641747456 00000100:00100000:2.0:1788845042.243112:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641747456 00000100:00100000:2.0:1788845042.243156:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1458 00000100:00100000:2.0:1788845042.243166:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641747456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.243188:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.243220:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae819 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d6b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.243258:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.243298:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023de300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660370688:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.243370:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.243480:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641747584 00000100:00100000:1.0:1788845042.243483:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641747584 00000100:00100000:1.0:1788845042.243512:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1050 00000100:00100000:1.0:1788845042.243517:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641747584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845042.243523:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845042.243531:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ae820 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d72 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.243556:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660370816:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845042.243586:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.246089:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023de300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660370688:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.246114:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:0.0:1788845042.246132:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae819 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d6b expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.246151:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.246154:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.246160:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287ae819 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d6b expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.246176:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:0.0:1788845042.246239:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641747456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3074us (3243us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.246247:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1458 00000100:00100000:1.0:1788845042.246569:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660370816:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845042.246592:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ae820 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d72 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.246609:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287ae820 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d72 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.246620:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:1.0:1788845042.246656:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641747584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3138us (3619us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845042.246664:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1050 00000100:00100000:3.0:1788845042.250220:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.250307:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.250432:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641747840 00000100:00100000:1.0:1788845042.250442:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641747840 00000100:00100000:1.0:1788845042.250507:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1051 00000100:00100000:1.0:1788845042.250517:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641747840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845042.250536:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845042.250554:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae827 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d80 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.250623:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660370944:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845042.250707:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.253113:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660370944:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845042.253155:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae827 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d80 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.253181:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae827 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d80 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.253194:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845042.253399:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2367:x1875739641747840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2882us (3089us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845042.253409:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1051 00000100:00100000:0.0:1788845042.253553:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641747712 00000100:00100000:0.0:1788845042.253571:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641747712 00000100:00100000:0.0:1788845042.253624:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1459 00000100:00100000:0.0:1788845042.253634:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641747712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.253652:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.253699:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae82e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d79 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.253738:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.253784:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660371072:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.253838:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6000 going to sleep for 11 seconds 00010000:00010000:2.0:1788845042.261467:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845042.261776:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383f800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660371072:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845042.261801:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:2.0:1788845042.261820:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae82e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d79 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.261839:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.261842:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845042.261847:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287ae82e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d79 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.261860:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:2.0:1788845042.261937:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641747712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8304us (11735us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.261945:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1459 00000100:00100000:2.0:1788845042.264329:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.264378:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.264408:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641748096 00000100:00100000:2.0:1788845042.264416:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641748096 00000100:00100000:2.0:1788845042.264469:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1052 00000100:00100000:2.0:1788845042.264479:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641748096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.264497:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.264514:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ae835 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d8e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.264583:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660371200:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.264654:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.265087:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641747968 00000100:00100000:3.0:1788845042.265092:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641747968 00000100:00100000:3.0:1788845042.265137:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1460 00000100:00100000:3.0:1788845042.265145:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641747968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.265160:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.265175:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae83c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d87 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.265207:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.265236:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660371328:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.265296:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.268673:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660371200:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845042.268726:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ae835 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d8e expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.268758:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287ae835 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d8e expref: 8 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.268772:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033a00, rc 0) 00000100:00100000:3.0:1788845042.268837:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641748096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4358us (4457us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.268846:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1052 00000100:00100000:3.0:1788845042.269538:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660371328:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.269672:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 640670 00010000:00010000:3.0:1788845042.269712:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae83c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d87 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.269729:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.269732:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.269737:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287ae83c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d87 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.269750:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:3.0:1788845042.269797:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641747968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4653us (5477us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.269805:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1460 00000100:00100000:0.0:1788845042.274524:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.274596:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641748352 00000100:00100000:3.0:1788845042.274606:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641748352 00000100:00100000:3.0:1788845042.274667:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1461 00000100:00100000:3.0:1788845042.274677:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641748352:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845042.274745:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845042.274772:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287ae786 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cd8 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.274791:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.274795:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845042.275045:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.275060:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.275068:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.275100:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 27 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.275119:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 27 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.275132:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660371456:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845042.275218:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.281409:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.281514:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660371456:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845042.281544:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.281626:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641748480 00000100:00100000:2.0:1788845042.281633:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641748480 00010000:00010000:2.0:1788845042.281670:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 28 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.281723:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 501 00000100:00100000:2.0:1788845042.281734:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:2370:x1875739641748480:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845042.281748:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845042.281755:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 28 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.281773:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 28 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.281794:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15200/0xb2187301287ae75c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cb5 expref: 28 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.281811:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.281816:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.281826:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.281835:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.281864:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845042.281952:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641748480:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 219us (552us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.281960:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 501 00010000:00010000:3.0:1788845042.282031:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.282040:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845042.282169:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967820 00000004:00010000:3.0:1788845042.282222:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 640670 -> 640670 00000004:00010000:3.0:1788845042.282226:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00000004:00010000:3.0:1788845042.282229:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845042 -> 1788845041 00000004:00010000:3.0:1788845042.282231:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00000004:00010000:3.0:1788845042.282233:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:3.0:1788845042.282240:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c762d922 reply state 00000000bb155a30 transno 4294967821 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.282249:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845042.282318:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641748352:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 7643us (7803us total) trans 4294967821 rc 0/0 00000100:00100000:3.0:1788845042.282326:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1461 00000020:00080000:3.0:1788845042.282331:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845042.284585:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.284601:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.284608:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.284614:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02031800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845042.284618:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.284626:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.284641:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.284644:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845042.284649:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.284655:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287ae843 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.286034:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.286240:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641748608 00000100:00100000:2.0:1788845042.286251:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641748608 00000100:00100000:2.0:1788845042.286321:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1462 00000100:00100000:2.0:1788845042.286332:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641748608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.286357:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.286375:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d95 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.286403:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.286413:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.286447:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.286464:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 26 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.286477:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660371584:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845042.286568:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.289182:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.289286:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641748736 00000100:00100000:1.0:1788845042.289295:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641748736 00002000:00100000:1.0:1788845042.289345:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964000 x1875739641748736/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.289357:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964000 with x1875739641748736 00010000:00010000:1.0:1788845042.289366:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.289385:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 27 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845042.289393:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f01964000 00000100:00100000:1.0:1788845042.289433:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 36 00000100:00100000:1.0:1788845042.289442:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641748736:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845042.289777:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660371584:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845042.289810:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.289826:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.289828:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845042.289832:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.289846:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032a00, rc 0) 00000100:00100000:2.0:1788845042.289878:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641748608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3547us (3852us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.289913:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1462 00000004:00010000:3.0:1788845042.293321:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 640670 -> 640670 00000004:00010000:3.0:1788845042.293329:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00000004:00010000:3.0:1788845042.293331:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:3.0:1788845042.293334:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00000004:00010000:3.0:1788845042.293336:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:3.0:1788845042.293410:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641748736:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3966us (4233us total) trans 4294967822 rc 0/0 00000100:00100000:3.0:1788845042.293423:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 36 00002000:00100000:3.0:1788845042.293437:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964000 x1875739641748736/t4294967822(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/448 e 0 to 0 dl 1788845053 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.293447:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964000 with x1875739641748736 00010000:00010000:3.0:1788845042.293457:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 29 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.293477:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 29 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845042.293485:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f01964000 00000100:00100000:1.0:1788845042.298314:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.298464:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641748864 00000100:00100000:2.0:1788845042.298474:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641748864 00010000:00010000:2.0:1788845042.298516:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 29 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.298561:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 502 00000100:00100000:2.0:1788845042.298680:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d10c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641748864:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845042.298705:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845042.298731:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 29 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.298749:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 29 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.298770:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287ae7a9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9cfb expref: 29 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.298785:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.298791:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.298805:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.298814:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.298853:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.298867:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660371712:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845042.299002:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.302364:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.302462:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660371712:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845042.302486:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845042.302549:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d10c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641748864:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3871us (4244us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.302559:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 502 00000100:00100000:3.0:1788845042.302828:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641748992 00000100:00100000:3.0:1788845042.302836:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641748992 00000100:00100000:3.0:1788845042.302917:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1053 00000100:00100000:3.0:1788845042.302929:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641748992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.302948:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.302969:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9d9c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.303005:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.303016:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(14 14) 00010000:00010000:3.0:1788845042.303021:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(15 14) 00010000:00010000:3.0:1788845042.303026:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.303063:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.303086:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 6 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.303099:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035689500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660371840:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845042.303161:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.306721:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035689500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660371840:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845042.306768:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.306792:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:3.0:1788845042.306857:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641748992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3930us (4501us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.306864:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1053 00000100:00100000:0.0:1788845042.308822:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.308913:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641749120 00000100:00100000:3.0:1788845042.308922:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641749120 00002000:00100000:3.0:1788845042.308974:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1740800->1748157) req@ffff949f01965500 x1875739641749120/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 7848/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.308986:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965500 with x1875739641749120 ext(1740800->1748157) 00010000:00010000:3.0:1788845042.308996:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 7 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.309015:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 7 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845042.309030:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f01965500 00000100:00100000:3.0:1788845042.309067:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 29 00000100:00100000:3.0:1788845042.309077:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641749120:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845042.309669:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641749120:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 591us (851us total) trans 4294967482 rc 0/0 00000100:00100000:3.0:1788845042.309678:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 29 00002000:00100000:3.0:1788845042.309696:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1740800->1748157) req@ffff949f01965500 x1875739641749120/t4294967482(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 7848/448 e 0 to 0 dl 1788845053 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.309707:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965500 with x1875739641749120 ext(1740800->1748157) 00010000:00010000:3.0:1788845042.309713:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 9 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.309727:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 9 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845042.309735:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f01965500 00000100:00100000:1.0:1788845042.312706:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.312821:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641749248 00000100:00100000:2.0:1788845042.312831:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641749248 00010000:00010000:2.0:1788845042.312867:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 9 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.313028:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 503 00000100:00100000:2.0:1788845042.313039:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641749248:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845042.313054:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845042.313076:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 9 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.313093:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 9 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.313114:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287ae7cc lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1740800->1748991) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d1e expref: 9 pid: 6685 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.313131:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(15 14) 00010000:00010000:2.0:1788845042.313135:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(15 14) 00010000:00010000:2.0:1788845042.313143:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.313152:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.313190:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 7 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.313213:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660371968:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845042.313307:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.316228:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.316284:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.316431:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641749376 00000100:00100000:0.0:1788845042.316439:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641749376 00002000:00010000:0.0:1788845042.316478:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845042.316483:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035689500 with x1875739641749376 00000100:00100000:0.0:1788845042.316523:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 965 00000100:00100000:0.0:1788845042.316532:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641749376:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845042.316933:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 640670 -> 2612 00000004:00010000:0.0:1788845042.316937:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00000004:00010000:0.0:1788845042.316940:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845042.316942:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00000004:00010000:0.0:1788845042.316944:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:0.0:1788845042.317007:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641749376:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 475us (787us total) trans 4294967823 rc 0/0 00000100:00100000:0.0:1788845042.317014:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 965 00002000:00010000:0.0:1788845042.317017:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845042.317020:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035689500 with x1875739641749376 00000100:00100000:2.0:1788845042.318238:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641749504 00000100:00100000:2.0:1788845042.318248:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641749504 00002000:00010000:2.0:1788845042.318292:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845042.318297:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03568a680 with x1875739641749504 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845042.318307:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03568a680. 00000100:00100000:2.0:1788845042.318331:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 487 00000100:00100000:2.0:1788845042.318340:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641749504:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845042.318601:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1748158 -> 0 00002000:00010000:2.0:1788845042.318605:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.318607:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.318609:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.318612:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 163 -> 163 00000100:00100000:2.0:1788845042.318653:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641749504:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 315us (2368us total) trans 4294967483 rc 0/0 00000100:00100000:2.0:1788845042.318659:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 487 00002000:00010000:2.0:1788845042.318662:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845042.318665:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03568a680 with x1875739641749504 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845042.318669:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03568a680. 00000100:00100000:2.0:1788845042.318725:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660371968:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845042.318742:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845042.318761:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641749248:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5725us (6076us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.318768:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 503 00000100:00100000:3.0:1788845042.322484:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.322593:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641749632 00000100:00100000:0.0:1788845042.322603:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641749632 00000100:00100000:0.0:1788845042.322662:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1463 00000100:00100000:0.0:1788845042.322672:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641749632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.322707:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.322729:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae858 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9da3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.322819:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.322834:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.322846:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845042.322873:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2612 00000004:00200000:0.0:1788845042.322877:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845042.322930:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.322960:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae858 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.322978:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 29 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.322986:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.322989:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845042.322993:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:0.0:1788845042.323049:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:14891:x1875739641749632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 379us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.323056:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1463 00000100:00100000:0.0:1788845042.325574:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.325703:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641749760 00000100:00100000:3.0:1788845042.325712:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641749760 00000100:00100000:3.0:1788845042.325773:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1464 00000100:00100000:3.0:1788845042.325783:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641749760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.325808:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.325828:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae866 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9daa expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.325860:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.325935:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035688380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660372096:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.326009:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.328311:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.328706:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641749888 00000100:00100000:2.0:1788845042.328715:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641749888 00000100:00100000:2.0:1788845042.328773:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1054 00000100:00100000:2.0:1788845042.328784:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641749888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.328802:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.328820:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ae86d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9db1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.328878:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660372224:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.328973:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788845042.334331:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845041 -> 1788845042 00000100:00100000:3.0:1788845042.334367:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035688380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660372096:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.334392:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 2612 00010000:00010000:3.0:1788845042.334411:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae866 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9daa expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.334432:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.334435:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.334440:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae866 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9daa expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.334456:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:3.0:1788845042.334517:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641749760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8734us (8961us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.334525:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1464 00000100:00100000:2.0:1788845042.336584:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660372224:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.336621:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ae86d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9db1 expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.336645:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2cf31a4/0xb2187301287ae86d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9db1 expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.336659:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033200, rc 0) 00000100:00100000:2.0:1788845042.336720:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641749888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7938us (8417us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.336729:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1054 00000100:00100000:1.0:1788845042.340414:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.340459:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.340549:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641750016 00000100:00100000:0.0:1788845042.340560:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641750016 00000100:00100000:0.0:1788845042.340619:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1465 00000100:00100000:0.0:1788845042.340629:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2367:x1875739641750016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.340653:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.340674:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae874 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9db8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.340723:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.340765:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660372352:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.340848:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.340977:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641750144 00000100:00100000:2.0:1788845042.340982:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641750144 00000100:00100000:2.0:1788845042.341022:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1055 00000100:00100000:2.0:1788845042.341030:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641750144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.341040:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.341050:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae87b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9dbf expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.341088:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660372480:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.341122:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.344483:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660372352:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.344511:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 2612 00010000:00010000:0.0:1788845042.344530:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae874 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9db8 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.344550:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.344553:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.344569:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae874 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9db8 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.344591:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845042.344659:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2367:x1875739641750016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4031us (4255us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.344668:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1465 00000100:00100000:2.0:1788845042.346769:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660372480:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.346814:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae87b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dbf expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.346838:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae87b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dbf expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.346852:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845042.346939:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641750144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5909us (6478us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.346948:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1055 00000100:00100000:3.0:1788845042.350062:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.350154:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641750272 00000100:00100000:0.0:1788845042.350164:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641750272 00000100:00100000:0.0:1788845042.350230:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1466 00000100:00100000:0.0:1788845042.350241:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+29:2368:x1875739641750272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.350275:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.350295:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae882 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9dc6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.350335:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.350375:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660372608:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.350459:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.351477:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.351593:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641750400 00000100:00100000:0.0:1788845042.351600:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641750400 00000100:00100000:0.0:1788845042.351645:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1056 00000100:00100000:0.0:1788845042.351654:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641750400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.351668:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.351691:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae889 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9dcd expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.351739:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660372736:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.351797:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.355951:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660372736:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845042.355994:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae889 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dcd expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.356021:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae889 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dcd expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.356034:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:3.0:1788845042.356094:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641750400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4440us (4619us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.356103:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1056 00000100:00100000:0.0:1788845042.357870:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660372608:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.358065:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 2612 00010000:00010000:0.0:1788845042.358088:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae882 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dc6 expref: 30 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.358109:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.358113:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.358118:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae882 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dc6 expref: 30 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.358132:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845042.358234:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+29:2368:x1875739641750272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7994us (8180us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.358244:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1466 00000100:00100000:3.0:1788845042.361152:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.361215:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.361518:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641750656 00000100:00100000:0.0:1788845042.361528:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641750656 00000100:00100000:0.0:1788845042.361594:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1057 00000100:00100000:0.0:1788845042.361605:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641750656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.361626:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.361645:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287ae890 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ddb expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.361713:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660372864:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.361791:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.363055:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641750528 00000100:00100000:2.0:1788845042.363065:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641750528 00000100:00100000:0.0:1788845042.363495:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1467 00000100:00100000:0.0:1788845042.363506:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:2370:x1875739641750528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.363525:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.363544:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae897 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9dd4 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.363590:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.363625:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660372992:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.363707:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.366872:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660372864:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845042.366955:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287ae890 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ddb expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.366980:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287ae890 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ddb expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.366995:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845042.367049:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641750656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5445us (5904us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.367057:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1057 00000100:00100000:3.0:1788845042.367647:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660372992:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.367661:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 2612 00010000:00010000:3.0:1788845042.367675:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae897 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dd4 expref: 30 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.367701:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.367704:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.367709:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287ae897 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dd4 expref: 30 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.367720:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006200, rc 0) 00000100:00100000:3.0:1788845042.367763:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:2370:x1875739641750528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4259us (6547us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.367771:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1467 00000100:00100000:1.0:1788845042.373544:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.373646:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641750784 00000100:00100000:2.0:1788845042.373657:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641750784 00000100:00100000:2.0:1788845042.373725:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1058 00000100:00100000:2.0:1788845042.373735:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641750784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.373753:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.373774:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9de2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.373806:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.373815:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(15 14) 00010000:00010000:2.0:1788845042.373819:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(16 14) 00010000:00010000:2.0:1788845042.373825:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1536000->1597439] (req 1536000->1597439) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.373860:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.373878:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 8 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.373930:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660373120:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845042.374008:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.376098:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660373120:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845042.376135:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1536000->1597439] (req 1536000->1597439) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.376155:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845042.376216:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641750784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2482us (2682us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.376225:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1058 00000100:00100000:1.0:1788845042.378035:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.378760:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641750912 00000100:00100000:0.0:1788845042.378771:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641750912 00010000:00010000:0.0:1788845042.378814:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 9 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.378929:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 504 00000100:00100000:0.0:1788845042.378940:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641750912:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845042.378958:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845042.378981:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 9 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.379000:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 9 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.379025:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000013d86cc7/0xb2187301287ae851 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d9c expref: 9 pid: 6684 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.379040:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(16 14) 00010000:00010000:0.0:1788845042.379045:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(16 14) 00010000:00010000:0.0:1788845042.379054:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.379064:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.379099:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.379118:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660373248:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845042.379192:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.382974:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660373248:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845042.383001:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845042.383048:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641750912:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4110us (5020us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.383060:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 504 00000100:00100000:1.0:1788845042.384747:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.384828:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641751040 00000100:00100000:3.0:1788845042.384837:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641751040 00000100:00100000:3.0:1788845042.384924:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1468 00000100:00100000:3.0:1788845042.384934:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2370:x1875739641751040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.384956:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.384978:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae8a5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9de9 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.385017:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.385058:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660373376:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.385130:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.387677:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660373376:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.387714:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 2612 00010000:00010000:3.0:1788845042.387733:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae8a5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de9 expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.387753:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.387756:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.387761:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae8a5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de9 expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.387775:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:3.0:1788845042.387834:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2370:x1875739641751040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2901us (3094us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.387842:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1468 00000100:00100000:3.0:1788845042.390149:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788845042.390171:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660373504:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845042.390247:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845042.390306:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660373504 00000100:00100000:3.0:1788845042.390314:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660373504 00000100:00100000:3.0:1788845042.390368:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 84 00000100:00100000:3.0:1788845042.390379:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035688000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660373504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788845042.390416:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788845042.390460:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035688000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660373504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (213us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.390468:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 84 00000100:00100000:3.0:1788845042.390513:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660373504:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1788845042.390605:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845042.390647:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.391078:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641751168 00000100:00100000:3.0:1788845042.391083:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641751168 00000100:00100000:3.0:1788845042.391122:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1469 00000100:00100000:3.0:1788845042.391130:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641751168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.391145:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.391167:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae8ac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9df0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.391309:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.391346:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660373632:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.391487:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.393872:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660373632:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.394007:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 2612 00010000:00010000:3.0:1788845042.394027:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae8ac lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9df0 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.394047:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.394050:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.394056:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae8ac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9df0 expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.394070:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845042.394226:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641751168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3095us (3579us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.394234:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1469 00000100:00100000:1.0:1788845042.397059:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.397148:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641751296 00000100:00100000:0.0:1788845042.397159:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641751296 00000100:00100000:0.0:1788845042.397447:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1470 00000100:00100000:0.0:1788845042.397459:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:14891:x1875739641751296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.397485:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.397507:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae8b3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9df7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.397613:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae8ba lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.397627:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae8ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.397638:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae8ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845042.397665:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2612 00000004:00200000:0.0:1788845042.397669:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845042.397699:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae8ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.397729:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287ae8b3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9df7 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.397749:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae8ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9df7 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.397757:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.397760:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845042.397764:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845042.397832:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641751296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 376us (785us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.397840:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1470 00000100:00100000:1.0:1788845042.400810:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.400943:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641751424 00000100:00100000:3.0:1788845042.400953:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641751424 00000100:00100000:3.0:1788845042.401047:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1059 00000100:00100000:3.0:1788845042.401059:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641751424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.401080:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.401100:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae8c1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9dfe expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.401169:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660373760:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.401284:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00002000:00010000:3.0:1788845042.403556:0:7983:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1594018 00000100:00100000:3.0:1788845042.403601:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660373760:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845042.403639:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae8c1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dfe expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.403664:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287ae8c1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9dfe expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.403679:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:3.0:1788845042.403754:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641751424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2696us (2954us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.403762:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1059 00000100:00100000:2.0:1788845042.408136:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.408318:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641751552 00000100:00100000:3.0:1788845042.408328:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641751552 00000100:00100000:3.0:1788845042.408386:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1060 00000100:00100000:3.0:1788845042.408395:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641751552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.408413:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.408432:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae8c8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e05 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.408494:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660373888:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.408730:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.411423:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660373888:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845042.411464:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae8c8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e05 expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.411491:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae8c8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e05 expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.411511:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845042.411692:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641751552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3287us (3556us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.411700:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1060 00000100:00100000:1.0:1788845042.415170:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.415369:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641751680 00000100:00100000:0.0:1788845042.415380:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641751680 00000100:00100000:0.0:1788845042.415470:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1061 00000100:00100000:0.0:1788845042.415481:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641751680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.415501:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.415521:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e0c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.415554:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.415575:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(16 14) 00010000:00010000:0.0:1788845042.415580:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(17 14) 00010000:00010000:0.0:1788845042.415585:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1241088->1265663] (req 1241088->1265663) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.415620:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 8 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.415638:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 8 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.415652:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660374016:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845042.415742:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0096d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.418935:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.419020:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641751808 00000100:00100000:3.0:1788845042.419029:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641751808 00002000:00100000:3.0:1788845042.419081:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1536000->1594017) req@ffff949f023de680 x1875739641751808/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.419092:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023de680 with x1875739641751808 ext(1536000->1594017) 00010000:00010000:3.0:1788845042.419101:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 9 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.419121:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 9 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845042.419128:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f023de680 00000100:00100000:3.0:1788845042.419163:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 30 00000100:00100000:3.0:1788845042.419172:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641751808:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845042.424218:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2370:x1875739641751808:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5036us (5282us total) trans 4294967484 rc 0/0 00000100:00100000:3.0:1788845042.424233:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 30 00002000:00100000:3.0:1788845042.424246:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1536000->1594017) req@ffff949f023de680 x1875739641751808/t4294967484(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/448 e 0 to 0 dl 1788845053 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.424264:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023de680 with x1875739641751808 ext(1536000->1594017) 00010000:00010000:3.0:1788845042.424275:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 11 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.424296:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 11 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845042.424304:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f023de680 00000100:00100000:0.0:1788845042.425351:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660374016:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845042.425387:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1241088->1265663] (req 1241088->1265663) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 9 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.425407:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845042.425450:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641751680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9970us (10288us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.425458:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1061 00000100:00100000:1.0:1788845042.429114:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.429327:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641751936 00000100:00100000:2.0:1788845042.429338:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641751936 00010000:00010000:2.0:1788845042.429378:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 11 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.429467:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 505 00000100:00100000:2.0:1788845042.429478:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641751936:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845042.429493:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845042.429516:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 11 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.429533:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 11 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.429569:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae89e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1536000->1597439) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9de2 expref: 11 pid: 6682 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.429587:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(17 14) 00010000:00010000:2.0:1788845042.429592:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(17 14) 00010000:00010000:2.0:1788845042.429601:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.429612:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.429652:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.429666:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023ddf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660374144:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845042.429753:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.435646:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.435705:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.435734:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641752064 00000100:00100000:2.0:1788845042.435744:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641752064 00002000:00100000:2.0:1788845042.435789:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1597439) req@ffff949f023ded80 x1875739641752064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.435800:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023ded80 with x1875739641752064 ext(1048576->1597439) 00002000:00010000:2.0:1788845042.435806:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023ded80 00000100:00100000:2.0:1788845042.435835:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 488 00000100:00100000:2.0:1788845042.435844:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641752064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.436304:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641752192 00000100:00100000:3.0:1788845042.436310:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641752192 00002000:00100000:3.0:1788845042.436339:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dca80 x1875739641752192/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.436349:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dca80 with x1875739641752192 00000100:00100000:3.0:1788845042.436371:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 966 00000100:00100000:3.0:1788845042.436378:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641752192:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.436756:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023ddf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660374144:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845042.436782:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845042.436809:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641751936:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7333us (7704us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.436818:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 505 00000100:00100000:3.0:1788845042.445503:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641752192:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 9113us (9856us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.445518:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 966 00002000:00100000:3.0:1788845042.445532:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dca80 x1875739641752192/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.445543:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dca80 with x1875739641752192 00000100:00100000:2.0:1788845042.450770:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641752064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14917us (15056us total) trans 0 rc 548864/548864 00000100:00100000:2.0:1788845042.450786:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 488 00002000:00100000:2.0:1788845042.450800:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1597439) req@ffff949f023ded80 x1875739641752064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 548864/548864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.450810:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023ded80 with x1875739641752064 ext(1048576->1597439) 00002000:00010000:2.0:1788845042.450817:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023ded80 00000100:00100000:1.0:1788845042.455242:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.455394:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641752448 00000100:00100000:2.0:1788845042.455406:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641752448 00000100:00100000:2.0:1788845042.455470:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1471 00000100:00100000:2.0:1788845042.455481:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641752448:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845042.455536:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845042.455575:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287ae8ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9df7 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.455601:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.455607:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845042.455829:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.455845:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.455854:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.455942:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 28 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.455964:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 28 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.455981:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023ddf80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660374272:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845042.456073:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.459493:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023ddf80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660374272:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845042.459526:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.460289:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.460368:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641752576 00000100:00100000:3.0:1788845042.460377:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641752576 00000100:00100000:3.0:1788845042.460435:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 285 00000100:00100000:3.0:1788845042.460446:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2370:x1875739641752576:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845042.460519:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2370:x1875739641752576:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 75us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.460526:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 285 00000100:00100000:1.0:1788845042.467808:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.467919:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641752704 00000100:00100000:2.0:1788845042.467929:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641752704 00010000:00010000:2.0:1788845042.467967:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 29 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.468006:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 506 00000100:00100000:2.0:1788845042.468017:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641752704:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845042.468033:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845042.468040:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 29 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.468058:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 29 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.468081:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287ae85f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9da3 expref: 29 pid: 11036 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.468098:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.468103:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.468114:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.468122:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.468158:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845042.468223:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+28:2369:x1875739641752704:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 206us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.468231:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 506 00010000:00010000:3.0:1788845042.468928:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.468938:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845042.469060:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967824 00000004:00010000:3.0:1788845042.469113:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 2612 -> 2612 00000004:00010000:3.0:1788845042.469117:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00000004:00010000:3.0:1788845042.469119:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845042 -> 1788845041 00000004:00010000:3.0:1788845042.469122:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00000004:00010000:3.0:1788845042.469124:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:3.0:1788845042.469132:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001d570647 reply state 00000000530113d9 transno 4294967825 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.469141:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845042.469240:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:14891:x1875739641752448:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 13759us (14008us total) trans 4294967825 rc 0/0 00000100:00100000:3.0:1788845042.469248:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1471 00000020:00080000:3.0:1788845042.469267:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845042.472928:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.472946:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.472953:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.472959:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02033600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845042.472963:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.472971:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.472989:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.472992:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845042.472997:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.473002:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287ae8d6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.474075:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.474299:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641752832 00000100:00100000:0.0:1788845042.474309:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641752832 00002000:00010000:0.0:1788845042.474347:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845042.474352:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20000 with x1875739641752832 00000100:00100000:0.0:1788845042.474383:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 967 00000100:00100000:0.0:1788845042.474394:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2370:x1875739641752832:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845042.474626:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 2612 -> 60281 00000004:00010000:0.0:1788845042.474630:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00000004:00010000:0.0:1788845042.474632:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:0.0:1788845042.474634:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00000004:00010000:0.0:1788845042.474637:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:0.0:1788845042.474719:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641752832:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 326us (652us total) trans 4294967826 rc 0/0 00000100:00100000:0.0:1788845042.474727:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 967 00002000:00010000:0.0:1788845042.474730:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845042.474733:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20000 with x1875739641752832 00000100:00100000:1.0:1788845042.477157:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.477262:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641752960 00000100:00100000:2.0:1788845042.477273:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641752960 00002000:00010000:2.0:1788845042.477313:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845042.477318:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f22680 with x1875739641752960 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845042.477329:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f22680. 00000100:00100000:2.0:1788845042.477362:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 489 00000100:00100000:2.0:1788845042.477372:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641752960:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845042.477714:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1594018 -> 0 00002000:00010000:2.0:1788845042.477717:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.477719:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.477721:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.477722:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 163 -> 163 00000100:00100000:2.0:1788845042.477771:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641752960:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 401us (623us total) trans 4294967485 rc 0/0 00000100:00100000:2.0:1788845042.477776:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 489 00002000:00010000:2.0:1788845042.477778:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845042.477780:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f22680 with x1875739641752960 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845042.477783:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f22680. 00000100:00100000:2.0:1788845042.481274:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.481380:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641753088 00000100:00100000:3.0:1788845042.481388:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641753088 00000100:00100000:3.0:1788845042.481438:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1472 00000100:00100000:3.0:1788845042.481445:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641753088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.481462:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.481477:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae8dd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e13 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.481546:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.481567:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.481575:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845042.481593:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 60281 00000004:00200000:3.0:1788845042.481596:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845042.481609:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.481627:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae8dd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.481640:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 29 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.481645:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.481648:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845042.481650:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:3.0:1788845042.481722:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:14891:x1875739641753088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 273us (465us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.481729:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1472 00000100:00100000:1.0:1788845042.483511:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.483546:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.483547:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641753216 00000100:00100000:3.0:1788845042.483555:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641753216 00000100:00100000:3.0:1788845042.483613:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1473 00000100:00100000:3.0:1788845042.483622:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641753216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.483639:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.483656:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae8eb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e1a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.483695:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.483728:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641753344 00000100:00100000:2.0:1788845042.483732:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641753344 00000100:00100000:3.0:1788845042.483740:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660374400:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.483768:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1062 00000100:00100000:2.0:1788845042.483775:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641753344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.483783:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.483794:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae8f2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e21 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.483819:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e220c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.483828:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660374528:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.483854:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000004:00010000:3.0:1788845042.485080:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845041 -> 1788845042 00000100:00100000:3.0:1788845042.485106:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660374400:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.485124:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 60281 00010000:00010000:3.0:1788845042.485139:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae8eb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e1a expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.485155:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00000100:00100000:2.0:1788845042.485157:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660374528:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845042.485158:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.485163:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae8eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e1a expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.485176:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae8f2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e21 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.485178:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00010000:00010000:2.0:1788845042.485195:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c819509/0xb2187301287ae8f2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e21 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.485217:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:3.0:1788845042.485222:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641753216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1601us (1717us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.485229:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1473 00000100:00100000:2.0:1788845042.485534:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641753344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1757us (1985us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.485541:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1062 00000100:00100000:1.0:1788845042.489661:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.489762:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.490196:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641753600 00000100:00100000:2.0:1788845042.490215:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641753600 00000100:00100000:2.0:1788845042.490352:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1063 00000100:00100000:2.0:1788845042.490363:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641753600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.490382:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.490401:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae8f9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e2f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.490464:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660374656:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.490707:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.490818:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641753472 00000100:00100000:3.0:1788845042.490823:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641753472 00000100:00100000:3.0:1788845042.490866:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1474 00000100:00100000:3.0:1788845042.490874:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2367:x1875739641753472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.490909:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.490923:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae900 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e28 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.490953:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.490977:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660374784:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.491005:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e220c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.493873:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03568b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660374784:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.493873:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660374656:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.493914:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 60281 00010000:00010000:3.0:1788845042.493927:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae900 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e28 expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.493927:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae8f9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e2f expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.493947:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.493949:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae8f9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e2f expref: 11 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.493950:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.493955:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae900 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e28 expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.493969:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00010000:00010000:2.0:1788845042.493970:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:3.0:1788845042.494061:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2367:x1875739641753472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3186us (4408us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.494070:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1474 00000100:00100000:2.0:1788845042.494094:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641753600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3734us (4333us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.494101:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1063 00000100:00100000:1.0:1788845042.496494:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.496610:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641753728 00000100:00100000:0.0:1788845042.496618:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641753728 00000100:00100000:0.0:1788845042.496665:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1475 00000100:00100000:0.0:1788845042.496675:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:14891:x1875739641753728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.496712:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.496730:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae907 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e36 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.496804:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae90e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.496816:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae90e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.496825:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae90e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845042.496843:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 60281 00000004:00200000:0.0:1788845042.496847:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845042.496861:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae90e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.496878:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae907 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e36 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.496966:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae90e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e36 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.496975:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.496978:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845042.496981:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845042.497028:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641753728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 354us (540us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.497035:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1475 00000100:00100000:1.0:1788845042.500815:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.501311:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641753984 00000100:00100000:0.0:1788845042.501322:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641753984 00000100:00100000:1.0:1788845042.501490:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1476 00000100:00100000:1.0:1788845042.501500:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:14891:x1875739641753984:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845042.501555:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845042.501597:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae90e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e36 expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.501617:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845042.501622:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845042.501846:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.501861:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.501874:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.502049:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 28 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.502069:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 28 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.502085:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660374912:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845042.502178:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.505399:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.505461:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641754112 00000100:00100000:0.0:1788845042.505469:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641754112 00010000:00010000:0.0:1788845042.505498:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 29 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.505530:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 507 00000100:00100000:0.0:1788845042.505537:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641754112:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845042.505547:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845042.505552:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 29 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.505576:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 29 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.505593:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.505597:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.505604:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.505609:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.505623:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845042.505666:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641754112:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 129us (276us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.505670:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 507 00010000:00010000:2.0:1788845042.506776:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287ae8e4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e13 expref: 28 pid: 5840 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.506797:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660374912:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845042.506819:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.506829:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.506833:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845042.507126:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967827 00000004:00010000:2.0:1788845042.507180:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 60281 -> 60281 00000004:00010000:2.0:1788845042.507184:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00000004:00010000:2.0:1788845042.507187:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845042 -> 1788845041 00000004:00010000:2.0:1788845042.507189:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00000004:00010000:2.0:1788845042.507192:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000004:00010000:2.0:1788845042.507209:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bfce7208 reply state 00000000e7a7bfd8 transno 4294967828 ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.507222:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845042.507335:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641753984:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5835us (6530us total) trans 4294967828 rc 0/0 00000100:00100000:2.0:1788845042.507342:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1476 00000020:00080000:2.0:1788845042.507347:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845042.508918:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.508934:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.508941:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.508947:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f01d15000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845042.508951:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.508964:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.508980:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.508983:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845042.508987:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.508993:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15000/0xb2187301287ae915 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.511168:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.511601:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641754240 00000100:00100000:2.0:1788845042.511611:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641754240 00002000:00010000:2.0:1788845042.511649:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1748917->18446744073709551615) 00002000:00100000:2.0:1788845042.511655:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f23800 with x1875739641754240 ext(1748917->18446744073709551615) 00002000:00010000:2.0:1788845042.511666:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f23800. 00000100:00100000:2.0:1788845042.511708:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 490 00000100:00100000:2.0:1788845042.511718:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641754240:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845042.511972:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1748917 00002000:00010000:2.0:1788845042.511976:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.511979:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.511981:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00002000:00010000:2.0:1788845042.511983:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 163 -> 163 00000100:00100000:2.0:1788845042.512088:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641754240:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 371us (928us total) trans 4294967486 rc 0/0 00000100:00100000:2.0:1788845042.512097:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 490 00002000:00010000:2.0:1788845042.512100:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1748917->18446744073709551615) 00002000:00100000:2.0:1788845042.512103:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f23800 with x1875739641754240 ext(1748917->18446744073709551615) 00002000:00010000:2.0:1788845042.512108:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f23800. 00000100:00100000:0.0:1788845042.516827:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.517004:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641754368 00000100:00100000:3.0:1788845042.517016:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641754368 00002000:00100000:3.0:1788845042.517067:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1748991) req@ffff949f01965f80 x1875739641754368/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.517079:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965f80 with x1875739641754368 ext(1597440->1748991) 00002000:00010000:3.0:1788845042.517084:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965f80 00000100:00100000:3.0:1788845042.517114:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 491 00000100:00100000:3.0:1788845042.517125:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641754368:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.526424:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641754368:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 9289us (9595us total) trans 0 rc 151552/151552 00000100:00100000:3.0:1788845042.526438:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 491 00002000:00100000:3.0:1788845042.526450:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1748991) req@ffff949f01965f80 x1875739641754368/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 151552/151552 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.526460:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01965f80 with x1875739641754368 ext(1597440->1748991) 00002000:00010000:3.0:1788845042.526466:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01965f80 00000100:00100000:1.0:1788845042.531747:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.531834:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641754496 00000100:00100000:2.0:1788845042.531844:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641754496 00002000:00100000:2.0:1788845042.531977:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1699840->1748916) req@ffff94a034f20a80 x1875739641754496/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.531989:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f20a80 with x1875739641754496 ext(1699840->1748916) 00002000:00010000:2.0:1788845042.531995:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f20a80 00000100:00100000:2.0:1788845042.532064:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 492 00000100:00100000:2.0:1788845042.532076:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641754496:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845042.539137:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641754496:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 7054us (7389us total) trans 4294967487 rc 0/0 00000100:00100000:0.0:1788845042.539152:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 492 00002000:00100000:0.0:1788845042.539166:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1699840->1748916) req@ffff94a034f20a80 x1875739641754496/t4294967487(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/448 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.539175:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a034f20a80 with x1875739641754496 ext(1699840->1748916) 00002000:00010000:0.0:1788845042.539181:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a034f20a80 00000100:00100000:2.0:1788845042.542569:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.542720:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641754624 00000100:00100000:0.0:1788845042.542729:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641754624 00000100:00100000:0.0:1788845042.542787:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1064 00000100:00100000:0.0:1788845042.542798:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641754624:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:3.0:1788845042.575826:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641754624:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 33018us (33266us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.575842:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1064 00000100:00100000:3.0:1788845042.578827:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.578973:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641754752 00000100:00100000:0.0:1788845042.578984:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641754752 00000100:00100000:0.0:1788845042.579047:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1477 00000100:00100000:0.0:1788845042.579057:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:14891:x1875739641754752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.579083:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.579103:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae91c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e3d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.579219:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae923 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.579235:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae923 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.579246:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae923 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845042.579277:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1748917 00000004:00200000:0.0:1788845042.579282:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845042.579302:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae923 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.579329:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287ae91c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e3d expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.579348:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae923 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e3d expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.579356:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.579359:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845042.579362:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845042.579424:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641754752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 369us (605us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.579432:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1477 00000001:00080000:1.0:1788845042.579757:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967487 is committed 00000001:00080000:1.0:1788845042.579787:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967484 is committed 00000100:00100000:0.0:1788845042.584571:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.584703:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641754880 00000100:00100000:3.0:1788845042.584714:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641754880 00000100:00100000:3.0:1788845042.584774:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1478 00000100:00100000:3.0:1788845042.584784:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+28:14891:x1875739641754880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.584808:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.584829:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae92a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e44 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.584956:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.584973:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.584984:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845042.585011:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1748917 00000004:00200000:3.0:1788845042.585016:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845042.585034:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.585064:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae92a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.585082:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 29 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.585090:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.585093:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845042.585097:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:3.0:1788845042.585167:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:14891:x1875739641754880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 384us (615us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.585174:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1478 00000100:00100000:0.0:1788845042.589118:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.589223:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641755008 00000100:00100000:3.0:1788845042.589233:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641755008 00000100:00100000:3.0:1788845042.589297:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1479 00000100:00100000:3.0:1788845042.589306:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641755008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.589329:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.589351:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae938 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e4b expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.589387:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.589430:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660375040:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.589513:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e223c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.592473:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.592751:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641755136 00000100:00100000:0.0:1788845042.592760:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641755136 00000100:00100000:0.0:1788845042.592820:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1065 00000100:00100000:0.0:1788845042.592831:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641755136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.592849:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.592866:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae93f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e52 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.592960:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660375168:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.593033:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4480 going to sleep for 11 seconds 00000004:00010000:3.0:1788845042.593524:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845041 -> 1788845042 00000100:00100000:3.0:1788845042.593547:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660375040:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.593577:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 60281 00010000:00010000:3.0:1788845042.593593:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae938 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e4b expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.593611:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.593614:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.593619:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae938 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e4b expref: 30 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.593639:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845042.593693:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641755008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4387us (4585us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.593701:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1479 00000100:00100000:0.0:1788845042.597404:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660375168:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845042.597447:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae93f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e52 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.597472:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287ae93f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e52 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.597485:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006800, rc 0) 00000100:00100000:0.0:1788845042.597549:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641755136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4719us (5084us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.597567:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1065 00000100:00100000:1.0:1788845042.600705:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.600748:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.600835:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641755392 00000100:00100000:2.0:1788845042.600846:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641755392 00000100:00100000:2.0:1788845042.600936:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1066 00000100:00100000:2.0:1788845042.600947:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641755392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.600967:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.600986:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ae946 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e60 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.601051:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660375296:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.601133:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.601532:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641755264 00000100:00100000:3.0:1788845042.601537:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641755264 00000100:00100000:3.0:1788845042.601582:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1480 00000100:00100000:3.0:1788845042.601590:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2370:x1875739641755264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.601602:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.601616:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae94d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e59 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.601646:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.601673:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660375424:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.601728:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e223c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.605220:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660375424:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.605248:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 60281 00010000:00010000:3.0:1788845042.605277:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae94d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e59 expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.605298:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.605301:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.605306:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae94d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e59 expref: 30 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.605322:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845042.605383:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+29:2370:x1875739641755264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3793us (4688us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.605392:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1480 00000100:00100000:2.0:1788845042.607068:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660375296:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.607102:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ae946 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e60 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.607123:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000281f0693/0xb2187301287ae946 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e60 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.607140:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033800, rc 0) 00000100:00100000:2.0:1788845042.607338:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641755392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6391us (6588us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.607347:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1066 00000100:00100000:1.0:1788845042.612024:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.612170:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641755520 00000100:00100000:2.0:1788845042.612180:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641755520 00002000:00100000:2.0:1788845042.612503:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f20e00 x1875739641755520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.612515:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20e00 with x1875739641755520 00000100:00100000:2.0:1788845042.612658:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 968 00000100:00100000:2.0:1788845042.612670:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641755520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.613011:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641755520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 342us (996us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.613019:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 968 00002000:00100000:2.0:1788845042.613027:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f20e00 x1875739641755520/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.613035:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20e00 with x1875739641755520 00000100:00100000:1.0:1788845042.616224:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.616654:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641755648 00000100:00100000:2.0:1788845042.616665:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641755648 00002000:00100000:2.0:1788845042.616721:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480380 x1875739641755648/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.616732:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480380 with x1875739641755648 00000100:00100000:2.0:1788845042.616851:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 969 00000100:00100000:2.0:1788845042.616862:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641755648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.617118:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641755648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 257us (903us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.617125:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 969 00002000:00100000:2.0:1788845042.617132:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480380 x1875739641755648/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.617139:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480380 with x1875739641755648 00000100:00100000:1.0:1788845042.620047:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.620086:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641755776 00000100:00100000:1.0:1788845042.620095:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641755776 00002000:00100000:1.0:1788845042.620145:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480700 x1875739641755776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.620162:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480700 with x1875739641755776 00000100:00100000:1.0:1788845042.620193:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 970 00000100:00100000:1.0:1788845042.620218:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641755776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.620455:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641755776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 238us (417us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845042.620463:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 970 00002000:00100000:1.0:1788845042.620470:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480700 x1875739641755776/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.620476:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480700 with x1875739641755776 00000100:00100000:1.0:1788845042.628601:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.628700:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641755904 00000100:00100000:3.0:1788845042.628710:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641755904 00002000:00100000:3.0:1788845042.628761:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481880 x1875739641755904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.628772:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481880 with x1875739641755904 00000100:00100000:3.0:1788845042.628807:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 971 00000100:00100000:3.0:1788845042.628817:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641755904:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.629104:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641755904:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 287us (512us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.629112:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 971 00002000:00100000:3.0:1788845042.629119:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481880 x1875739641755904/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.629126:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481880 with x1875739641755904 00000100:00100000:1.0:1788845042.632732:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.632835:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641756032 00000100:00100000:2.0:1788845042.632847:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641756032 00002000:00100000:2.0:1788845042.632939:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481180 x1875739641756032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.632951:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481180 with x1875739641756032 00000100:00100000:2.0:1788845042.632980:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 972 00000100:00100000:2.0:1788845042.632992:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641756032:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.633294:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641756032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 304us (571us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.633303:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 972 00002000:00100000:2.0:1788845042.633310:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02481180 x1875739641756032/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.633318:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02481180 with x1875739641756032 00000100:00100000:2.0:1788845042.637086:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.637177:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641756160 00000100:00100000:3.0:1788845042.637188:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641756160 00002000:00100000:3.0:1788845042.637250:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023de300 x1875739641756160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.637262:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023de300 with x1875739641756160 00000100:00100000:3.0:1788845042.637302:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 973 00000100:00100000:3.0:1788845042.637313:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641756160:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.637607:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641756160:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 294us (531us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.637615:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 973 00002000:00100000:3.0:1788845042.637622:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023de300 x1875739641756160/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.637630:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023de300 with x1875739641756160 00000100:00100000:1.0:1788845042.641393:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.641573:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641756288 00000100:00100000:2.0:1788845042.641583:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641756288 00002000:00100000:2.0:1788845042.641632:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482d80 x1875739641756288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.641648:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482d80 with x1875739641756288 00000100:00100000:2.0:1788845042.641695:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 974 00000100:00100000:2.0:1788845042.641704:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641756288:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.641984:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641756288:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (600us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.641992:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 974 00002000:00100000:2.0:1788845042.641999:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02482d80 x1875739641756288/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.642006:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02482d80 with x1875739641756288 00000100:00100000:2.0:1788845042.645626:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.645774:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641756416 00000100:00100000:3.0:1788845042.645785:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641756416 00002000:00100000:3.0:1788845042.645833:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023df100 x1875739641756416/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.645844:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023df100 with x1875739641756416 00000100:00100000:3.0:1788845042.645919:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 975 00000100:00100000:3.0:1788845042.645930:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641756416:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.646187:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641756416:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 258us (571us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.646194:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 975 00002000:00100000:3.0:1788845042.646725:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023df100 x1875739641756416/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.646736:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023df100 with x1875739641756416 00000100:00100000:1.0:1788845042.649775:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.649922:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641756544 00000100:00100000:2.0:1788845042.649934:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641756544 00002000:00100000:2.0:1788845042.649991:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480000 x1875739641756544/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.650004:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480000 with x1875739641756544 00000100:00100000:2.0:1788845042.650044:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 976 00000100:00100000:2.0:1788845042.650053:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641756544:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.650512:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641756544:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 457us (745us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.650520:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 976 00002000:00100000:2.0:1788845042.650527:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02480000 x1875739641756544/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.650534:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02480000 with x1875739641756544 00000100:00100000:0.0:1788845042.654280:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.654428:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641756672 00000100:00100000:1.0:1788845042.654439:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641756672 00002000:00100000:1.0:1788845042.654487:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966680 x1875739641756672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.654499:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966680 with x1875739641756672 00000100:00100000:1.0:1788845042.654538:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 977 00000100:00100000:1.0:1788845042.654548:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641756672:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.654818:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641756672:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 269us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845042.654826:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 977 00002000:00100000:1.0:1788845042.654833:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966680 x1875739641756672/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.654841:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966680 with x1875739641756672 00000100:00100000:0.0:1788845042.658990:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.659100:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641756800 00000100:00100000:2.0:1788845042.659110:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641756800 00002000:00100000:2.0:1788845042.659160:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964a80 x1875739641756800/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.659172:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964a80 with x1875739641756800 00000100:00100000:2.0:1788845042.659224:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 978 00000100:00100000:2.0:1788845042.659235:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641756800:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.659506:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641756800:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 272us (527us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.659514:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 978 00002000:00100000:2.0:1788845042.659522:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964a80 x1875739641756800/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.659529:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964a80 with x1875739641756800 00000100:00100000:2.0:1788845042.664669:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.664785:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641756928 00000100:00100000:1.0:1788845042.664796:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641756928 00002000:00100000:1.0:1788845042.664847:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9ad80 x1875739641756928/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.664859:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9ad80 with x1875739641756928 00000100:00100000:1.0:1788845042.664933:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 979 00000100:00100000:1.0:1788845042.664943:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641756928:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845042.669286:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 60281 -> 602112 00000004:00010000:1.0:1788845042.669297:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00000004:00010000:1.0:1788845042.669299:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845042 -> 1788845041 00000004:00010000:1.0:1788845042.669301:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00000004:00010000:1.0:1788845042.669303:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 584 00000100:00100000:1.0:1788845042.669394:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2367:x1875739641756928:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4446us (4729us total) trans 4294967829 rc 0/0 00000100:00100000:1.0:1788845042.669407:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 979 00002000:00100000:1.0:1788845042.669420:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01f9ad80 x1875739641756928/t4294967829(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/448 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.669429:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01f9ad80 with x1875739641756928 00000100:00100000:0.0:1788845042.674170:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.674317:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641757056 00000100:00100000:3.0:1788845042.674328:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641757056 00000100:00100000:3.0:1788845042.674387:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1481 00000100:00100000:3.0:1788845042.674399:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2368:x1875739641757056:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845042.716339:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967826 is committed 00000001:00080000:1.0:1788845042.716362:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967813 is committed 00000001:00080000:1.0:1788845042.716377:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967829 is committed 00000001:00080000:1.0:1788845042.716408:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967822 is committed 00000100:00100000:0.0:1788845042.716705:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641757056:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 42305us (42540us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845042.716717:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1481 00000100:00100000:0.0:1788845042.720958:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.721516:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641757184 00000100:00100000:2.0:1788845042.721526:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641757184 00000100:00100000:2.0:1788845042.721597:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1482 00000100:00100000:2.0:1788845042.721608:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641757184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.721629:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.721652:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e67 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.721692:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.721701:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.721736:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.721753:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 10 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.721767:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d92a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660375552:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845042.721938:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.726063:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.726552:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d92a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660375552:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845042.726598:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.726618:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.726621:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.726626:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.726636:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032e00, rc 0) 00000100:00100000:3.0:1788845042.726707:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641757184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5100us (5761us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.726715:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1482 00000100:00100000:0.0:1788845042.727395:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641757312 00000100:00100000:0.0:1788845042.727404:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641757312 00010000:00010000:0.0:1788845042.727437:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 11 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.727471:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 508 00000100:00100000:0.0:1788845042.727481:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641757312:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845042.727493:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845042.727518:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 11 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.727535:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 11 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.727669:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032a00/0xb2187301287ae84a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9d95 expref: 11 pid: 5839 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.727694:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.727700:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.727710:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.727724:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.727760:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.727774:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660375680:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845042.727956:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ccc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.733514:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.733590:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641757440 00000100:00100000:3.0:1788845042.733600:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641757440 00000100:00100000:3.0:1788845042.733671:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 980 00000100:00100000:3.0:1788845042.733694:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641757440:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845042.734030:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 602112 -> 602112 00000004:00010000:3.0:1788845042.734034:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845042 00000004:00010000:3.0:1788845042.734037:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:3.0:1788845042.734040:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845042 00000004:00010000:3.0:1788845042.734042:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 584 -> 198 00000100:00100000:3.0:1788845042.734120:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641757440:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 428us (613us total) trans 4294967830 rc 0/0 00000100:00100000:3.0:1788845042.734129:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 980 00000100:00100000:3.0:1788845042.737697:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.737908:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641757568 00000100:00100000:0.0:1788845042.737919:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641757568 00000100:00100000:0.0:1788845042.737982:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1483 00000100:00100000:0.0:1788845042.737993:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641757568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.738014:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.738035:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae95b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e6e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.738074:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.738122:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b5180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660375808:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.738218:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000004:00010000:0.0:1788845042.740927:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845041 -> 1788845042 00000100:00100000:0.0:1788845042.740968:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b5180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660375808:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.740991:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:0.0:1788845042.741010:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae95b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e6e expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.741032:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.741035:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.741040:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287ae95b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e6e expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.741055:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:0.0:1788845042.741162:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641757568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3170us (3483us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.741171:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1483 00000100:00100000:2.0:1788845042.741397:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660375680:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845042.741410:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845042.741434:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641757312:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 13955us (15382us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845042.741443:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 508 00000100:00100000:0.0:1788845042.744715:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.744786:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641757696 00000100:00100000:3.0:1788845042.744796:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641757696 00000100:00100000:3.0:1788845042.744860:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1484 00000100:00100000:3.0:1788845042.744871:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641757696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.744911:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.744932:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae962 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e75 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.744973:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.745012:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660375936:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.745762:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.748081:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660375936:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.748109:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:3.0:1788845042.748128:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae962 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e75 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.748149:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.748153:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.748157:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae962 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e75 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.748171:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845042.748235:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641757696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3366us (3530us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.748244:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1484 00000100:00100000:1.0:1788845042.751621:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.751724:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641757824 00000100:00100000:3.0:1788845042.751735:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641757824 00000100:00100000:3.0:1788845042.751827:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1485 00000100:00100000:3.0:1788845042.751839:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641757824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.751861:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.751905:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae969 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.751949:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.751992:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660376064:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.752073:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845042.755015:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660376064:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845042.755043:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:1.0:1788845042.755061:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae969 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e7c expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.755083:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845042.755086:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845042.755092:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287ae969 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e7c expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.755108:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:1.0:1788845042.755174:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641757824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3337us (3629us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845042.755182:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1485 00000100:00100000:3.0:1788845042.758282:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.758370:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641757952 00000100:00100000:2.0:1788845042.758381:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641757952 00000100:00100000:2.0:1788845042.758438:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1486 00000100:00100000:2.0:1788845042.758448:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641757952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.758469:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.758492:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032600/0xb2187301287ae970 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e83 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.758529:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.758585:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660376192:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.758662:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.761367:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660376192:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.761395:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:3.0:1788845042.761414:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032600/0xb2187301287ae970 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e83 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.761435:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.761438:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.761442:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032600/0xb2187301287ae970 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e83 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.761458:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032600, rc 0) 00000100:00100000:3.0:1788845042.761517:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641757952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3069us (3242us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.761525:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1486 00000100:00100000:2.0:1788845042.768437:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.768519:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641758080 00000100:00100000:3.0:1788845042.768530:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641758080 00000100:00100000:3.0:1788845042.768596:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1067 00000100:00100000:3.0:1788845042.768607:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641758080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.768629:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.768654:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae977 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e8a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.768723:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660376320:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.768800:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.771576:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660376320:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845042.771613:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae977 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e8a expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.771638:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae977 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e8a expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.771651:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845042.771716:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641758080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3111us (3289us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.771724:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1067 00000100:00100000:2.0:1788845042.775991:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.776113:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641758208 00000100:00100000:0.0:1788845042.776124:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641758208 00000100:00100000:0.0:1788845042.776181:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1068 00000100:00100000:0.0:1788845042.776190:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641758208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.776211:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.776231:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae97e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e91 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.776303:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660376448:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.776394:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.778458:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660376448:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845042.778495:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae97e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e91 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.778524:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287ae97e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e91 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.778536:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:0.0:1788845042.778603:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641758208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2412us (2620us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.778610:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1068 00000100:00100000:3.0:1788845042.782383:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.782465:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641758336 00000100:00100000:0.0:1788845042.782476:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641758336 00000100:00100000:0.0:1788845042.782539:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1487 00000100:00100000:0.0:1788845042.782550:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641758336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.782580:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.782601:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae985 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e98 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.782638:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.782687:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660376576:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.782766:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.785001:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660376576:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.785025:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:0.0:1788845042.785043:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae985 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e98 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.785071:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.785074:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.785078:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287ae985 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e98 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.785093:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845042.785149:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641758336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2601us (2778us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.785157:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1487 00000100:00100000:2.0:1788845042.788346:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.788429:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641758464 00000100:00100000:3.0:1788845042.788437:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641758464 00002000:00100000:3.0:1788845042.788480:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1449983) req@ffff949f0251c380 x1875739641758464/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.788497:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251c380 with x1875739641758464 ext(1445888->1449983) 00002000:00010000:3.0:1788845042.788502:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251c380 00000100:00100000:3.0:1788845042.788531:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 493 00000100:00100000:3.0:1788845042.788540:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641758464:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.788781:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641758464:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 239us (440us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.788789:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 493 00002000:00100000:3.0:1788845042.788796:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1449983) req@ffff949f0251c380 x1875739641758464/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.788803:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0251c380 with x1875739641758464 ext(1445888->1449983) 00002000:00010000:3.0:1788845042.788807:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0251c380 00000100:00100000:3.0:1788845042.792344:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.792990:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641758592 00000100:00100000:0.0:1788845042.793002:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641758592 00002000:00100000:0.0:1788845042.793051:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1449984->1454079) req@ffff94a0345d2300 x1875739641758592/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.793061:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0345d2300 with x1875739641758592 ext(1449984->1454079) 00002000:00010000:0.0:1788845042.793067:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0345d2300 00000100:00100000:0.0:1788845042.793102:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 494 00000100:00100000:0.0:1788845042.793112:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641758592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.793520:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641758592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 408us (1184us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.793529:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 494 00002000:00100000:0.0:1788845042.793536:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1449984->1454079) req@ffff94a0345d2300 x1875739641758592/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.793544:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0345d2300 with x1875739641758592 ext(1449984->1454079) 00002000:00010000:0.0:1788845042.793548:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0345d2300 00000100:00100000:1.0:1788845042.797130:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.797221:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641758720 00000100:00100000:3.0:1788845042.797232:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641758720 00002000:00100000:3.0:1788845042.797279:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1454080->1458175) req@ffff949f02481f80 x1875739641758720/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.797290:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02481f80 with x1875739641758720 ext(1454080->1458175) 00002000:00010000:3.0:1788845042.797295:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02481f80 00000100:00100000:3.0:1788845042.797331:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 495 00000100:00100000:3.0:1788845042.797341:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641758720:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.797584:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641758720:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (463us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.797592:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 495 00002000:00100000:3.0:1788845042.797599:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1454080->1458175) req@ffff949f02481f80 x1875739641758720/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.797607:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02481f80 with x1875739641758720 ext(1454080->1458175) 00002000:00010000:3.0:1788845042.797611:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02481f80 00000100:00100000:2.0:1788845042.801155:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.801297:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641758848 00000100:00100000:0.0:1788845042.801307:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641758848 00002000:00100000:0.0:1788845042.801358:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1458176->1462271) req@ffff94a0360ed180 x1875739641758848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.801370:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ed180 with x1875739641758848 ext(1458176->1462271) 00002000:00010000:0.0:1788845042.801376:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ed180 00000100:00100000:0.0:1788845042.801414:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 496 00000100:00100000:0.0:1788845042.801423:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641758848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.801683:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641758848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (527us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.801691:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 496 00002000:00100000:0.0:1788845042.801698:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1458176->1462271) req@ffff94a0360ed180 x1875739641758848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.801705:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ed180 with x1875739641758848 ext(1458176->1462271) 00002000:00010000:0.0:1788845042.801709:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ed180 00000100:00100000:0.0:1788845042.806448:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.806538:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641758976 00000100:00100000:3.0:1788845042.806548:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641758976 00002000:00100000:3.0:1788845042.806616:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1462272->1466367) req@ffff94a0360b4700 x1875739641758976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.806627:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b4700 with x1875739641758976 ext(1462272->1466367) 00002000:00010000:3.0:1788845042.806633:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b4700 00000100:00100000:3.0:1788845042.806666:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 497 00000100:00100000:3.0:1788845042.806685:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641758976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.806997:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641758976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 322us (558us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.807004:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 497 00002000:00100000:3.0:1788845042.807011:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1462272->1466367) req@ffff94a0360b4700 x1875739641758976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.807018:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b4700 with x1875739641758976 ext(1462272->1466367) 00002000:00010000:3.0:1788845042.807021:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b4700 00000100:00100000:3.0:1788845042.810327:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.810713:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641759104 00000100:00100000:0.0:1788845042.810723:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641759104 00002000:00100000:0.0:1788845042.810771:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1466368->1470463) req@ffff94a0345d1180 x1875739641759104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.810783:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0345d1180 with x1875739641759104 ext(1466368->1470463) 00002000:00010000:0.0:1788845042.810789:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0345d1180 00000100:00100000:0.0:1788845042.810824:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 498 00000100:00100000:0.0:1788845042.810833:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641759104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.811257:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641759104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 423us (937us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.811268:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 498 00002000:00100000:0.0:1788845042.811275:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1466368->1470463) req@ffff94a0345d1180 x1875739641759104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.811284:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0345d1180 with x1875739641759104 ext(1466368->1470463) 00002000:00010000:0.0:1788845042.811287:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0345d1180 00000100:00100000:0.0:1788845042.817347:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.817437:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641759232 00000100:00100000:3.0:1788845042.817446:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641759232 00002000:00100000:3.0:1788845042.817495:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1470464->1474559) req@ffff94a0360b6d80 x1875739641759232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.817512:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b6d80 with x1875739641759232 ext(1470464->1474559) 00002000:00010000:3.0:1788845042.817518:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b6d80 00000100:00100000:3.0:1788845042.817657:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 499 00000100:00100000:3.0:1788845042.817667:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641759232:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.817946:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641759232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (606us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.817954:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 499 00002000:00100000:3.0:1788845042.817961:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1470464->1474559) req@ffff94a0360b6d80 x1875739641759232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.817970:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b6d80 with x1875739641759232 ext(1470464->1474559) 00002000:00010000:3.0:1788845042.817974:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b6d80 00000100:00100000:2.0:1788845042.821320:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.821431:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641759360 00000100:00100000:1.0:1788845042.821441:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641759360 00002000:00100000:1.0:1788845042.821489:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1474560->1478655) req@ffff94a0360ed500 x1875739641759360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.821500:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ed500 with x1875739641759360 ext(1474560->1478655) 00002000:00010000:1.0:1788845042.821505:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ed500 00000100:00100000:1.0:1788845042.821537:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 500 00000100:00100000:1.0:1788845042.821546:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641759360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845042.822045:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641759360:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 498us (732us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845042.822053:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 500 00002000:00100000:1.0:1788845042.822060:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1474560->1478655) req@ffff94a0360ed500 x1875739641759360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.822069:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ed500 with x1875739641759360 ext(1474560->1478655) 00002000:00010000:1.0:1788845042.822073:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ed500 00000100:00100000:1.0:1788845042.825585:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.825724:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641759488 00000100:00100000:2.0:1788845042.825734:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641759488 00002000:00100000:2.0:1788845042.825779:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1478656->1482751) req@ffff949f02483480 x1875739641759488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.825789:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02483480 with x1875739641759488 ext(1478656->1482751) 00002000:00010000:2.0:1788845042.825795:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02483480 00000100:00100000:2.0:1788845042.825828:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 501 00000100:00100000:2.0:1788845042.825838:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641759488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.826118:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641759488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (541us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845042.826126:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 501 00002000:00100000:2.0:1788845042.826136:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1478656->1482751) req@ffff949f02483480 x1875739641759488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.826145:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02483480 with x1875739641759488 ext(1478656->1482751) 00002000:00010000:2.0:1788845042.826149:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02483480 00000100:00100000:0.0:1788845042.829475:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.829707:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641759616 00000100:00100000:2.0:1788845042.829718:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641759616 00002000:00100000:2.0:1788845042.829759:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1482752->1486847) req@ffff94a0360b7b80 x1875739641759616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.829769:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7b80 with x1875739641759616 ext(1482752->1486847) 00002000:00010000:2.0:1788845042.829774:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7b80 00000100:00100000:2.0:1788845042.829805:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 502 00000100:00100000:2.0:1788845042.829813:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641759616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845042.830105:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641759616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 291us (637us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845042.830112:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 502 00002000:00100000:2.0:1788845042.830119:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1482752->1486847) req@ffff94a0360b7b80 x1875739641759616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.830128:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7b80 with x1875739641759616 ext(1482752->1486847) 00002000:00010000:2.0:1788845042.830131:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7b80 00000100:00100000:2.0:1788845042.833454:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.833524:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641759744 00000100:00100000:3.0:1788845042.833534:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641759744 00002000:00100000:3.0:1788845042.833590:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1486848->1490943) req@ffff94a0360edf80 x1875739641759744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.833601:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360edf80 with x1875739641759744 ext(1486848->1490943) 00002000:00010000:3.0:1788845042.833607:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360edf80 00000100:00100000:3.0:1788845042.833640:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 503 00000100:00100000:3.0:1788845042.833649:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641759744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.834161:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641759744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 510us (713us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.834168:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 503 00002000:00100000:3.0:1788845042.834175:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1486848->1490943) req@ffff94a0360edf80 x1875739641759744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.834183:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360edf80 with x1875739641759744 ext(1486848->1490943) 00002000:00010000:3.0:1788845042.834187:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360edf80 00000100:00100000:2.0:1788845042.837525:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.837645:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641759872 00000100:00100000:0.0:1788845042.837655:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641759872 00002000:00100000:0.0:1788845042.837715:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1490944->1495039) req@ffff94a0360eed80 x1875739641759872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.837726:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360eed80 with x1875739641759872 ext(1490944->1495039) 00002000:00010000:0.0:1788845042.837731:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360eed80 00000100:00100000:0.0:1788845042.837766:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 504 00000100:00100000:0.0:1788845042.837776:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641759872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.838074:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641759872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 299us (557us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.838083:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 504 00002000:00100000:0.0:1788845042.838090:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1490944->1495039) req@ffff94a0360eed80 x1875739641759872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.838098:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360eed80 with x1875739641759872 ext(1490944->1495039) 00002000:00010000:0.0:1788845042.838102:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360eed80 00000100:00100000:0.0:1788845042.841742:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.841846:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641760000 00000100:00100000:1.0:1788845042.841856:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641760000 00002000:00100000:1.0:1788845042.841925:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1495039) req@ffff94a0360b5880 x1875739641760000/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845042.841936:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5880 with x1875739641760000 ext(1445888->1495039) 00002000:00010000:1.0:1788845042.841942:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5880 00000100:00100000:1.0:1788845042.842010:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 505 00000100:00100000:1.0:1788845042.842020:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641760000:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845042.846537:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641760000:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4508us (4794us total) trans 4294967488 rc 0/0 00000100:00100000:2.0:1788845042.846560:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 505 00002000:00100000:2.0:1788845042.846573:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1495039) req@ffff94a0360b5880 x1875739641760000/t4294967488(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/448 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845042.846582:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b5880 with x1875739641760000 ext(1445888->1495039) 00002000:00010000:2.0:1788845042.846587:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b5880 00000100:00100000:1.0:1788845042.849099:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.849201:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641760128 00000100:00100000:2.0:1788845042.849212:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641760128 00000100:00100000:2.0:1788845042.849265:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1069 00000100:00100000:2.0:1788845042.849276:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641760128:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845042.874636:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967488 is committed 00000100:00100000:1.0:1788845042.874831:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641760128:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 25552us (25735us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845042.874842:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1069 00000100:00100000:0.0:1788845042.878171:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.878331:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641760256 00000100:00100000:2.0:1788845042.878340:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641760256 00000100:00100000:2.0:1788845042.878432:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1070 00000100:00100000:2.0:1788845042.878442:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641760256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.878461:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.878478:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000393941f5/0xb2187301287ae98c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9e9f expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.878535:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660376704:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.878737:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788845042.880963:0:6682:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 163 -> 203 00000100:00100000:1.0:1788845042.880995:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660376704:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845042.881033:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000393941f5/0xb2187301287ae98c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e9f expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.881058:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000393941f5/0xb2187301287ae98c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e9f expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845042.881077:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033400, rc 0) 00000100:00100000:1.0:1788845042.881178:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641760256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2737us (3016us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845042.881188:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1070 00000100:00100000:1.0:1788845042.884957:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.885257:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641760384 00000100:00100000:3.0:1788845042.885271:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641760384 00000100:00100000:3.0:1788845042.885339:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1071 00000100:00100000:3.0:1788845042.885351:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641760384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.885374:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.885393:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae993 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ea6 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.885458:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660376832:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.885564:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e229c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.888568:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660376832:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845042.888608:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae993 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ea6 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.888631:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae993 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ea6 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.888644:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845042.888715:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641760384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3363us (3767us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.888726:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1071 00000100:00100000:2.0:1788845042.891639:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.891731:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641760512 00000100:00100000:3.0:1788845042.891741:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641760512 00000100:00100000:3.0:1788845042.891799:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1488 00000100:00100000:3.0:1788845042.891810:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641760512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.891831:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.891850:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae99a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ead expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.891915:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.891958:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660376960:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.892034:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e229c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.894700:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660376960:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.894725:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:3.0:1788845042.894744:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae99a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ead expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.894765:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.894768:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.894773:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287ae99a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ead expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.894788:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845042.894847:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641760512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3038us (3217us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.894856:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1488 00000100:00100000:1.0:1788845042.897625:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.897709:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641760640 00000100:00100000:3.0:1788845042.897720:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641760640 00000100:00100000:3.0:1788845042.897780:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1489 00000100:00100000:3.0:1788845042.897791:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641760640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.897811:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.897831:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae9a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9eb4 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.897868:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.897953:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660377088:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.898037:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e229c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.900630:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660377088:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845042.900656:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:3.0:1788845042.900684:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae9a1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9eb4 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.900705:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845042.900708:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845042.900713:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287ae9a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9eb4 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.900728:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845042.900787:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641760640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2998us (3171us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.900796:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1489 00000100:00100000:2.0:1788845042.905582:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.905717:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641760768 00000100:00100000:0.0:1788845042.905728:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641760768 00002000:00100000:0.0:1788845042.905775:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1142784->1146879) req@ffff94a0360ee680 x1875739641760768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.905787:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ee680 with x1875739641760768 ext(1142784->1146879) 00002000:00010000:0.0:1788845042.905793:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ee680 00000100:00100000:0.0:1788845042.905827:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 506 00000100:00100000:0.0:1788845042.905837:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641760768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.906100:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641760768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 264us (526us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.906107:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 506 00002000:00100000:0.0:1788845042.906113:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1142784->1146879) req@ffff94a0360ee680 x1875739641760768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.906121:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ee680 with x1875739641760768 ext(1142784->1146879) 00002000:00010000:0.0:1788845042.906124:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ee680 00000100:00100000:1.0:1788845042.909177:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.909392:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641760896 00000100:00100000:3.0:1788845042.909402:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641760896 00002000:00100000:3.0:1788845042.909453:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1171456->1175551) req@ffff949f02483b80 x1875739641760896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.909466:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02483b80 with x1875739641760896 ext(1171456->1175551) 00002000:00010000:3.0:1788845042.909472:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02483b80 00000100:00100000:3.0:1788845042.909509:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 507 00000100:00100000:3.0:1788845042.909518:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641760896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.909770:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641760896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 252us (601us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.909778:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 507 00002000:00100000:3.0:1788845042.909785:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1171456->1175551) req@ffff949f02483b80 x1875739641760896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.909793:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02483b80 with x1875739641760896 ext(1171456->1175551) 00002000:00010000:3.0:1788845042.909796:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02483b80 00000100:00100000:3.0:1788845042.913118:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.913230:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641761024 00000100:00100000:2.0:1788845042.913240:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641761024 00000100:00100000:2.0:1788845042.913301:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1490 00000100:00100000:2.0:1788845042.913312:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641761024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.913336:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.913358:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae9a8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ebb expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.913394:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.913434:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660377216:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.913522:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.915752:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660377216:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845042.915779:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:2.0:1788845042.915800:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae9a8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ebb expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.915819:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845042.915822:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845042.915826:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033c00/0xb2187301287ae9a8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ebb expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.915841:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033c00, rc 0) 00000100:00100000:2.0:1788845042.915940:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641761024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2630us (2832us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.915948:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1490 00000100:00100000:1.0:1788845042.919182:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.919296:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641761152 00000100:00100000:0.0:1788845042.919306:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641761152 00000100:00100000:0.0:1788845042.919366:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1491 00000100:00100000:0.0:1788845042.919376:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641761152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.919398:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.919419:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae9af lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ec2 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.919459:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.919500:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b7480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660377344:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.919813:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.923214:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b7480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660377344:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845042.923243:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:0.0:1788845042.923276:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae9af lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ec2 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.923300:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845042.923303:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845042.923309:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287ae9af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ec2 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.923326:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845042.923382:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641761152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4006us (4208us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.923391:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1491 00000100:00100000:1.0:1788845042.927377:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.927459:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641761280 00000100:00100000:0.0:1788845042.927469:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641761280 00002000:00100000:0.0:1788845042.927516:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1421312->1425407) req@ffff949f02482a00 x1875739641761280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.927527:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02482a00 with x1875739641761280 ext(1421312->1425407) 00002000:00010000:0.0:1788845042.927532:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02482a00 00000100:00100000:0.0:1788845042.927575:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 508 00000100:00100000:0.0:1788845042.927585:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641761280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.927839:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641761280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (471us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.927846:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 508 00002000:00100000:0.0:1788845042.927853:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1421312->1425407) req@ffff949f02482a00 x1875739641761280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.927861:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02482a00 with x1875739641761280 ext(1421312->1425407) 00002000:00010000:0.0:1788845042.927865:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02482a00 00000100:00100000:1.0:1788845042.931368:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.931533:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641761408 00000100:00100000:3.0:1788845042.931543:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641761408 00002000:00100000:3.0:1788845042.931603:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1425408->1429503) req@ffff94a03beab480 x1875739641761408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.931614:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beab480 with x1875739641761408 ext(1425408->1429503) 00002000:00010000:3.0:1788845042.931619:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beab480 00000100:00100000:3.0:1788845042.931653:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 509 00000100:00100000:3.0:1788845042.931663:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641761408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.931967:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641761408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 303us (606us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.931974:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 509 00002000:00100000:3.0:1788845042.931982:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1425408->1429503) req@ffff94a03beab480 x1875739641761408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.931989:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beab480 with x1875739641761408 ext(1425408->1429503) 00002000:00010000:3.0:1788845042.931993:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beab480 00000100:00100000:3.0:1788845042.935404:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.935505:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641761536 00000100:00100000:0.0:1788845042.935516:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641761536 00002000:00100000:0.0:1788845042.935577:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1429504->1433599) req@ffff94a0345d0700 x1875739641761536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.935588:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0345d0700 with x1875739641761536 ext(1429504->1433599) 00002000:00010000:0.0:1788845042.935593:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0345d0700 00000100:00100000:0.0:1788845042.935629:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 510 00000100:00100000:0.0:1788845042.935639:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641761536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.935923:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641761536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 283us (527us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.935932:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 510 00002000:00100000:0.0:1788845042.935938:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1429504->1433599) req@ffff94a0345d0700 x1875739641761536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.935946:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0345d0700 with x1875739641761536 ext(1429504->1433599) 00002000:00010000:0.0:1788845042.935949:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0345d0700 00000100:00100000:1.0:1788845042.938426:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.938515:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641761664 00000100:00100000:3.0:1788845042.938525:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641761664 00002000:00100000:3.0:1788845042.938577:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1433600->1437695) req@ffff94a03beabb80 x1875739641761664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.938588:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beabb80 with x1875739641761664 ext(1433600->1437695) 00002000:00010000:3.0:1788845042.938593:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beabb80 00000100:00100000:3.0:1788845042.938627:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 511 00000100:00100000:3.0:1788845042.938637:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641761664:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.938874:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641761664:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 238us (455us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.938903:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 511 00002000:00100000:3.0:1788845042.938912:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1433600->1437695) req@ffff94a03beabb80 x1875739641761664/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.938919:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beabb80 with x1875739641761664 ext(1433600->1437695) 00002000:00010000:3.0:1788845042.938922:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beabb80 00000100:00100000:1.0:1788845042.941840:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.941928:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641761792 00000100:00100000:0.0:1788845042.941937:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641761792 00002000:00100000:0.0:1788845042.941977:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1437696->1441791) req@ffff94a03beaa680 x1875739641761792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.941989:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beaa680 with x1875739641761792 ext(1437696->1441791) 00002000:00010000:0.0:1788845042.941995:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beaa680 00000100:00100000:0.0:1788845042.942024:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 512 00000100:00100000:0.0:1788845042.942032:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beaa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641761792:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845042.942373:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beaa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641761792:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 341us (542us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845042.942381:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 512 00002000:00100000:0.0:1788845042.942388:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1437696->1441791) req@ffff94a03beaa680 x1875739641761792/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845042.942396:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03beaa680 with x1875739641761792 ext(1437696->1441791) 00002000:00010000:0.0:1788845042.942399:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03beaa680 00000100:00100000:0.0:1788845042.944269:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.944376:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641761920 00000100:00100000:3.0:1788845042.944384:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641761920 00002000:00100000:3.0:1788845042.944419:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1441792->1445887) req@ffff94a0360b7480 x1875739641761920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.944430:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7480 with x1875739641761920 ext(1441792->1445887) 00002000:00010000:3.0:1788845042.944435:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7480 00000100:00100000:3.0:1788845042.944462:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 513 00000100:00100000:3.0:1788845042.944470:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641761920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.944645:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641761920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 174us (382us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845042.944652:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 513 00002000:00100000:3.0:1788845042.944658:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1441792->1445887) req@ffff94a0360b7480 x1875739641761920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:148/0 lens 488/4536 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.944666:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360b7480 with x1875739641761920 ext(1441792->1445887) 00002000:00010000:3.0:1788845042.944669:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360b7480 00000100:00100000:1.0:1788845042.949378:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845042.949482:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641762048 00000100:00100000:0.0:1788845042.949492:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641762048 00000100:00100000:0.0:1788845042.949820:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1072 00000100:00100000:0.0:1788845042.949831:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641762048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845042.949850:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845042.949868:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae9b6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ec9 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845042.949967:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660377472:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845042.950102:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845042.952911:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660377472:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845042.952951:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae9b6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ec9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.952978:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287ae9b6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ec9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845042.952995:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845042.953093:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641762048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3264us (3723us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845042.953103:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1072 00000100:00100000:0.0:1788845042.956096:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.956374:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641762176 00000100:00100000:1.0:1788845042.956387:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641762176 00000100:00100000:1.0:1788845042.956487:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1073 00000100:00100000:1.0:1788845042.956497:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845042.956514:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845042.956532:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae9bd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ed0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.956602:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660377600:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845042.956697:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.960126:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660377600:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.960171:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae9bd lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ed0 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.960330:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287ae9bd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ed0 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.960345:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:2.0:1788845042.960439:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3942us (4350us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.960448:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1073 00000100:00100000:0.0:1788845042.964165:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845042.964373:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641762304 00000100:00100000:1.0:1788845042.964383:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641762304 00000100:00100000:1.0:1788845042.964445:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1074 00000100:00100000:1.0:1788845042.964456:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641762304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845042.964475:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845042.964495:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ae9c4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ed7 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845042.964567:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660377728:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845042.964647:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.967479:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660377728:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.967522:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ae9c4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ed7 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.967561:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1f49/0xb2187301287ae9c4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ed7 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.967575:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15200, rc 0) 00000100:00100000:2.0:1788845042.967637:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641762304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3181us (3482us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.967646:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1074 00000100:00100000:2.0:1788845042.971093:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.971171:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641762432 00000100:00100000:3.0:1788845042.971182:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641762432 00000100:00100000:3.0:1788845042.971248:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1075 00000100:00100000:3.0:1788845042.971268:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.971288:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.971309:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae9cb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ede expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.971367:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660377856:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.971440:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.975212:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660377856:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.975264:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae9cb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ede expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.975289:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287ae9cb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ede expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.975303:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:2.0:1788845042.975366:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4109us (4282us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.975375:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1075 00000100:00100000:0.0:1788845042.979565:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.979706:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641762560 00000100:00100000:3.0:1788845042.979715:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641762560 00000100:00100000:3.0:1788845042.979773:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1076 00000100:00100000:3.0:1788845042.979783:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845042.979801:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845042.979820:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9d2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ee5 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845042.979880:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660377984:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845042.979992:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845042.982292:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660377984:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845042.982332:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9d2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ee5 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.982357:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9d2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ee5 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845042.982371:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845042.982430:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2648us (2889us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845042.982438:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1076 00000100:00100000:0.0:1788845042.985801:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845042.985920:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641762688 00000100:00100000:2.0:1788845042.985929:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641762688 00000100:00100000:2.0:1788845042.985982:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1077 00000100:00100000:2.0:1788845042.985993:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845042.986012:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845042.986035:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae9d9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9eec expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845042.986096:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660378112:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845042.986179:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845042.988768:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660378112:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845042.988807:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae9d9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9eec expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.988831:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae9d9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9eec expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845042.988843:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845042.988928:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2935us (3134us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845042.988936:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1077 00000100:00100000:1.0:1788845042.991930:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845042.992008:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641762816 00000100:00100000:3.0:1788845042.992017:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641762816 00002000:00100000:3.0:1788845042.992065:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea9180 x1875739641762816/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/0 e 0 to 0 dl 1788845053 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.992075:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea9180 with x1875739641762816 00000100:00100000:3.0:1788845042.992114:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 981 00000100:00100000:3.0:1788845042.992123:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641762816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845042.992372:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641762816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 249us (486us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845042.992379:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 981 00002000:00100000:3.0:1788845042.992387:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea9180 x1875739641762816/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:148/0 lens 488/440 e 0 to 0 dl 1788845053 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845042.992393:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea9180 with x1875739641762816 00000100:00100000:0.0:1788845043.000284:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.000378:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641762944 00000100:00100000:2.0:1788845043.000390:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641762944 00000100:00100000:2.0:1788845043.000469:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1078 00000100:00100000:2.0:1788845043.000481:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.000502:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.000520:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae9e0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ef3 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.000586:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660378240:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.000694:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.003394:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660378240:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.003430:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae9e0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ef3 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.003452:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000669cc588/0xb2187301287ae9e0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ef3 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.003465:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033e00, rc 0) 00000100:00100000:2.0:1788845043.003516:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641762944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3038us (3242us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.003524:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1078 00000100:00100000:1.0:1788845043.006109:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.006189:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641763072 00000100:00100000:3.0:1788845043.006206:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641763072 00002000:00100000:3.0:1788845043.006276:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beab100 x1875739641763072/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.006287:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beab100 with x1875739641763072 00000100:00100000:3.0:1788845043.006325:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 982 00000100:00100000:3.0:1788845043.006335:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641763072:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.006631:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641763072:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 297us (530us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.006638:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 982 00002000:00100000:3.0:1788845043.006645:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beab100 x1875739641763072/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.006652:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beab100 with x1875739641763072 00000100:00080000:1.0:1788845043.007355:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845043.010736:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.010826:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641763200 00000100:00100000:3.0:1788845043.010835:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641763200 00000100:00100000:3.0:1788845043.010938:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1079 00000100:00100000:3.0:1788845043.010948:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641763200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.010970:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.010991:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9e7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9efa expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.011053:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660378368:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.011137:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00080000:1.0:1788845043.011222:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845043.014479:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660378368:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.014518:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9e7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9efa expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.014541:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9e7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9efa expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.014566:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845043.014632:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641763200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3686us (3905us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.014641:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1079 00000100:00100000:0.0:1788845043.017823:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.017931:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641763328 00000100:00100000:3.0:1788845043.017943:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641763328 00002000:00100000:3.0:1788845043.017990:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036080380 x1875739641763328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.018000:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036080380 with x1875739641763328 00000100:00100000:3.0:1788845043.018038:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 983 00000100:00100000:3.0:1788845043.018047:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641763328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.018317:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641763328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 270us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.018324:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 983 00002000:00100000:3.0:1788845043.018332:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036080380 x1875739641763328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.018339:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036080380 with x1875739641763328 00000100:00100000:0.0:1788845043.021773:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.021912:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641763456 00000100:00100000:3.0:1788845043.021922:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641763456 00000100:00100000:3.0:1788845043.021981:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1080 00000100:00100000:3.0:1788845043.021992:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641763456:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.022011:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.022030:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9ee lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f01 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.022089:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660378496:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.022174:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.024432:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660378496:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.024471:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9ee lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f01 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.024496:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287ae9ee lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f01 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.024511:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845043.024580:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641763456:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2590us (2818us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.024589:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1080 00000100:00100000:0.0:1788845043.027190:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.027378:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641763584 00000100:00100000:3.0:1788845043.027388:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641763584 00002000:00100000:3.0:1788845043.027436:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082d80 x1875739641763584/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.027446:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082d80 with x1875739641763584 00000100:00100000:3.0:1788845043.027482:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 984 00000100:00100000:3.0:1788845043.027491:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641763584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.027757:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641763584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (573us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.027764:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 984 00002000:00100000:3.0:1788845043.027771:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082d80 x1875739641763584/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.027778:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082d80 with x1875739641763584 00000100:00100000:0.0:1788845043.032122:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.032221:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641763712 00000100:00100000:2.0:1788845043.032231:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641763712 00000100:00100000:2.0:1788845043.032299:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1081 00000100:00100000:2.0:1788845043.032309:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641763712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.032328:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.032347:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287ae9f5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f08 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.032411:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660378624:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.032499:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.035119:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660378624:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845043.035160:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287ae9f5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f08 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.035187:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b79328b2/0xb2187301287ae9f5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f08 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.035213:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032800, rc 0) 00000100:00100000:1.0:1788845043.035272:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641763712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2964us (3160us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.035280:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1081 00000100:00100000:0.0:1788845043.038403:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.038493:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641763840 00000100:00100000:2.0:1788845043.038503:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641763840 00000100:00100000:2.0:1788845043.038567:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1082 00000100:00100000:2.0:1788845043.038578:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641763840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.038596:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.038615:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae9fc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f0f expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.038690:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660378752:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.038774:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.041202:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660378752:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.041243:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae9fc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f0f expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.041279:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287ae9fc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f0f expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.041293:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845043.041351:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641763840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2775us (2957us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.041360:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1082 00000100:00100000:0.0:1788845043.044588:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.044723:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641763968 00000100:00100000:3.0:1788845043.044733:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641763968 00000100:00100000:3.0:1788845043.044794:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1083 00000100:00100000:3.0:1788845043.044804:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641763968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.044823:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.044840:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aea03 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f16 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.045033:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660378880:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.045112:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.047503:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660378880:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.047542:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aea03 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f16 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.047579:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aea03 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f16 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.047592:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845043.047647:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641763968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2843us (3067us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.047655:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1083 00000100:00100000:0.0:1788845043.050716:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.051017:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641764096 00000100:00100000:3.0:1788845043.051028:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641764096 00000100:00100000:2.0:1788845043.051227:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1084 00000100:00100000:2.0:1788845043.051239:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641764096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.051274:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.051296:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287aea0a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f1d expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.051355:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660379008:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.051436:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.053727:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660379008:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.053765:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287aea0a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f1d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.053791:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287aea0a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f1d expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.053805:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845043.053863:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641764096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2626us (3156us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.053871:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1084 00000100:00100000:0.0:1788845043.056935:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.057027:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641764224 00000100:00100000:3.0:1788845043.057037:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641764224 00000100:00100000:3.0:1788845043.057111:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1492 00000100:00100000:3.0:1788845043.057121:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641764224:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.057141:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.057161:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aea11 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f24 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.057210:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.057253:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660379136:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.057326:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00080000:0.0:1788845043.057383:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00010000:3.0:1788845043.059639:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845042 -> 1788845043 00000100:00100000:3.0:1788845043.059683:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660379136:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.059706:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:3.0:1788845043.059724:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aea11 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f24 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.059743:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.059746:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.059751:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aea11 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f24 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.059765:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030600, rc 0) 00000100:00100000:3.0:1788845043.059821:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641764224:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2701us (2894us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.059830:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1492 00000100:00100000:0.0:1788845043.062539:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.062848:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641764352 00000100:00100000:3.0:1788845043.062858:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641764352 00000100:00100000:3.0:1788845043.062938:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1493 00000100:00100000:3.0:1788845043.062949:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641764352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.062969:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.062990:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea18 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f2b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.063028:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.063075:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660379264:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.063224:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.066126:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660379264:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.066154:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:3.0:1788845043.066173:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea18 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f2b expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.066409:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.066412:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.066419:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f2b expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.066435:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845043.066537:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641764352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3588us (4007us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.066555:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1493 00000100:00100000:1.0:1788845043.070117:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.070342:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641764480 00000100:00100000:2.0:1788845043.070353:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641764480 00000100:00100000:2.0:1788845043.070445:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1085 00000100:00100000:2.0:1788845043.070457:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641764480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.070479:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.070499:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287aea1f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f32 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.070572:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660379392:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.070707:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845043.073362:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845042 -> 1788845043 00000100:00100000:2.0:1788845043.073393:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660379392:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.073428:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287aea1f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f32 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.073452:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddea14ce/0xb2187301287aea1f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f32 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.073466:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033600, rc 0) 00000100:00100000:2.0:1788845043.073643:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641764480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3187us (3535us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.073651:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1085 00000100:00100000:0.0:1788845043.079029:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.079127:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641764608 00000100:00100000:1.0:1788845043.079138:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641764608 00000100:00100000:1.0:1788845043.079369:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1086 00000100:00100000:1.0:1788845043.079382:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641764608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.079401:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.079420:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287aea26 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f39 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.079484:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660379520:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.079680:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.082439:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660379520:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.082482:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287aea26 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f39 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.082508:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c30f242/0xb2187301287aea26 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f39 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.082523:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:2.0:1788845043.082717:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641764608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3337us (3841us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.082725:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1086 00000100:00100000:0.0:1788845043.085702:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.085804:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641764736 00000100:00100000:3.0:1788845043.085814:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641764736 00000100:00100000:3.0:1788845043.085936:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1087 00000100:00100000:3.0:1788845043.085947:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641764736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.085966:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.085984:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aea2d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f40 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.086045:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660379648:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.086186:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.090122:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660379648:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.090165:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aea2d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f40 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.090318:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aea2d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f40 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.090336:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845043.090431:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641764736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4485us (4739us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.090440:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1087 00000100:00100000:1.0:1788845043.098129:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.098347:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641764864 00000100:00100000:3.0:1788845043.098358:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641764864 00002000:00100000:3.0:1788845043.098410:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea8e00 x1875739641764864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.098423:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea8e00 with x1875739641764864 00000100:00100000:3.0:1788845043.098499:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 985 00000100:00100000:3.0:1788845043.098512:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641764864:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.098862:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641764864:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 353us (746us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.098871:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 985 00002000:00100000:3.0:1788845043.098880:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea8e00 x1875739641764864/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.098920:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea8e00 with x1875739641764864 00000100:00100000:0.0:1788845043.102765:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.102849:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641764992 00000100:00100000:3.0:1788845043.102859:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641764992 00002000:00100000:3.0:1788845043.102926:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082680 x1875739641764992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.102937:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082680 with x1875739641764992 00000100:00100000:3.0:1788845043.103007:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 986 00000100:00100000:3.0:1788845043.103018:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641764992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.103310:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641764992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 292us (554us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.103318:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 986 00002000:00100000:3.0:1788845043.103325:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036082680 x1875739641764992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.103333:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082680 with x1875739641764992 00000100:00100000:0.0:1788845043.107056:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.107187:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641765120 00000100:00100000:3.0:1788845043.107317:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641765120 00000100:00100000:3.0:1788845043.107413:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1494 00000100:00100000:3.0:1788845043.107424:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641765120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.107446:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.107466:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea34 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f47 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.107504:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.107626:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660379776:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.107721:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.109811:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660379776:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.109834:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:3.0:1788845043.109853:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea34 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f47 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.109873:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.109876:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.109880:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea34 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f47 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.109928:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845043.109981:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641765120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2559us (2934us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.109988:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1494 00000100:00100000:1.0:1788845043.112856:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.113025:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641765248 00000100:00100000:0.0:1788845043.113037:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641765248 00000100:00100000:0.0:1788845043.113124:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1495 00000100:00100000:0.0:1788845043.113136:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641765248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.113157:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.113179:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aea3b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f4e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.113229:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.113282:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660379904:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.113399:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.116663:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660379904:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.116699:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 602112 00010000:00010000:3.0:1788845043.116720:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aea3b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f4e expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.116740:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.116743:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.116747:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aea3b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f4e expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.116766:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:3.0:1788845043.116862:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641765248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3728us (4016us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.116870:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1495 00000100:00100000:1.0:1788845043.119836:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.120437:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641765504 00000100:00100000:0.0:1788845043.120452:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641765504 00000100:00100000:2.0:1788845043.120746:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1496 00000100:00100000:2.0:1788845043.120757:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641765504:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845043.120809:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845043.120836:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287ae923 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e3d expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.120856:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.120861:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845043.121091:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.121106:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.121114:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.121147:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.121165:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 15 pid: 5840 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.121178:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660380032:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845043.121499:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.125212:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.125295:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641765632 00000100:00100000:3.0:1788845043.125306:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641765632 00010000:00010000:3.0:1788845043.125348:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 16 pid: 5840 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.125389:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 509 00000100:00100000:3.0:1788845043.125399:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d11450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641765632:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845043.125413:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845043.125419:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 16 pid: 5840 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.125437:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 16 pid: 5840 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.125458:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.125464:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.125473:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.125480:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.125501:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845043.126207:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287ae931 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e44 expref: 16 pid: 5840 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.126227:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ed880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660380032:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845043.126247:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.126266:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.126273:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845043.126390:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967831 00000004:00010000:2.0:1788845043.126440:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 602112 -> 602112 00000004:00010000:2.0:1788845043.126444:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845042 -> 1788845043 00000004:00010000:2.0:1788845043.126446:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845041 00000004:00010000:2.0:1788845043.126448:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845042 -> 1788845043 00000004:00010000:2.0:1788845043.126450:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 198 -> 198 00000004:00010000:2.0:1788845043.126457:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007e155ce7 reply state 000000000c5ad66e transno 4294967832 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.126466:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845043.127001:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d11450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641765632:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 1600us (1797us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.127011:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 509 00000100:00100000:2.0:1788845043.127488:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641765504:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6729us (7658us total) trans 4294967832 rc 0/0 00000100:00100000:2.0:1788845043.127495:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1496 00000020:00080000:2.0:1788845043.127500:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845043.131075:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.131147:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641765760 00000100:00100000:3.0:1788845043.131157:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641765760 00000100:00100000:3.0:1788845043.131228:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1497 00000100:00100000:3.0:1788845043.131238:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641765760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.131262:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.131282:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f55 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.131314:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.131325:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.131361:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.131379:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 14 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.131392:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660380160:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845043.131472:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.134204:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.134397:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641765888 00000100:00100000:0.0:1788845043.134406:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641765888 00010000:00010000:0.0:1788845043.134447:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.134488:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 510 00000100:00100000:0.0:1788845043.134498:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641765888:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845043.134512:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845043.134537:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.134564:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.134584:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.134590:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.134601:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.134610:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.134648:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.134662:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660380288:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845043.134744:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312e40 going to sleep for 11 seconds 00010000:00010000:3.0:1788845043.135435:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032e00/0xb2187301287ae954 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e67 expref: 15 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.135453:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0345d1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660380160:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845043.135475:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.135488:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.135491:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.135494:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845043.135538:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641765760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4301us (4471us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.135554:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1497 00010000:00010000:2.0:1788845043.138407:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.138428:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.138436:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.138442:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02033600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845043.138447:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.138455:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.138472:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.138475:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845043.138479:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.138485:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033600/0xb2187301287aea42 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.138798:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660380288:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845043.138819:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845043.138905:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641765888:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4382us (4696us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.138917:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 510 00000100:00100000:1.0:1788845043.141975:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.142087:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641766016 00000100:00100000:3.0:1788845043.142098:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641766016 00002000:00010000:3.0:1788845043.142137:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845043.142142:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea9c00 with x1875739641766016 00000100:00100000:3.0:1788845043.142180:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 987 00000100:00100000:3.0:1788845043.142204:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641766016:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845043.142400:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 602112 -> 912867 00000004:00010000:3.0:1788845043.142403:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.142406:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:3.0:1788845043.142408:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.142411:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 198 -> 198 00000100:00100000:3.0:1788845043.142486:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641766016:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 297us (519us total) trans 4294967833 rc 0/0 00000100:00100000:3.0:1788845043.142497:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 987 00002000:00010000:3.0:1788845043.142501:0:5854:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845043.142503:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea9c00 with x1875739641766016 00000100:00100000:2.0:1788845043.145145:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.145438:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641766144 00000100:00100000:1.0:1788845043.145448:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641766144 00002000:00010000:1.0:1788845043.145508:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845043.145513:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ed880 with x1875739641766144 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845043.145523:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ed880. 00000100:00100000:1.0:1788845043.145566:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 514 00000100:00100000:1.0:1788845043.145576:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641766144:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845043.146045:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1748917 -> 0 00002000:00010000:1.0:1788845043.146050:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845042 -> 1788845043 00002000:00010000:1.0:1788845043.146053:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845043 -> 1788845042 00002000:00010000:1.0:1788845043.146054:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845042 -> 1788845043 00002000:00010000:1.0:1788845043.146057:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 203 -> 203 00000100:00100000:1.0:1788845043.146123:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641766144:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 548us (985us total) trans 4294967489 rc 0/0 00000100:00100000:1.0:1788845043.146132:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 514 00002000:00010000:1.0:1788845043.146136:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845043.146139:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ed880 with x1875739641766144 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845043.146145:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ed880. 00000100:00080000:0.0:1788845043.146554:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845043.149812:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.150109:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641766272 00000100:00100000:3.0:1788845043.150119:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641766272 00000100:00100000:3.0:1788845043.150176:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1498 00000100:00100000:3.0:1788845043.150186:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641766272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.150224:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.150246:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aea50 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f5c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.150331:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.150347:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.150360:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845043.150389:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 912867 00000004:00200000:3.0:1788845043.150393:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845043.150412:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.150439:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aea50 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.150457:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.150474:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.150476:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845043.150479:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:3.0:1788845043.150553:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641766272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 360us (742us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.150560:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1498 00000100:00100000:3.0:1788845043.159233:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.159351:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641766400 00000100:00100000:0.0:1788845043.159361:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641766400 00002000:00100000:0.0:1788845043.159412:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d1f80 x1875739641766400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.159425:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d1f80 with x1875739641766400 00000100:00100000:0.0:1788845043.159465:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 988 00000100:00100000:0.0:1788845043.159475:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641766400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845043.171794:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641766400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 12310us (12560us total) trans 0 rc 913408/913408 00000100:00100000:1.0:1788845043.171808:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 988 00002000:00100000:1.0:1788845043.171821:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0345d1f80 x1875739641766400/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 913408/913408 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.171836:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0345d1f80 with x1875739641766400 00000100:00100000:3.0:1788845043.180987:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.181094:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641766528 00000100:00100000:1.0:1788845043.181104:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641766528 00002000:00100000:1.0:1788845043.181154:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90700 x1875739641766528/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.181165:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90700 with x1875739641766528 00000100:00100000:1.0:1788845043.181343:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 989 00000100:00100000:1.0:1788845043.181358:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641766528:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845043.184777:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 912867 -> 912867 00000004:00010000:1.0:1788845043.184789:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00000004:00010000:1.0:1788845043.184792:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845041 -> 1788845041 00000004:00010000:1.0:1788845043.184794:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00000004:00010000:1.0:1788845043.184797:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 198 -> 198 00000100:00100000:1.0:1788845043.184878:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641766528:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3521us (3896us total) trans 4294967834 rc 0/0 00000100:00100000:1.0:1788845043.184913:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 989 00002000:00100000:1.0:1788845043.184927:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90700 x1875739641766528/t4294967834(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/448 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.184937:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90700 with x1875739641766528 00000100:00100000:3.0:1788845043.188987:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.189067:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641766656 00000100:00100000:1.0:1788845043.189078:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641766656 00000100:00100000:1.0:1788845043.189138:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1499 00000100:00100000:1.0:1788845043.189148:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641766656:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:2.0:1788845043.226057:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967834 is committed 00000001:00080000:2.0:1788845043.226087:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967830 is committed 00000100:00100000:0.0:1788845043.226487:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641766656:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 37335us (37503us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.226499:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1499 00000100:00100000:0.0:1788845043.229499:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.229659:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641766784 00000100:00100000:3.0:1788845043.229679:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641766784 00000100:00100000:3.0:1788845043.229774:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1500 00000100:00100000:3.0:1788845043.229785:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641766784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.229806:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.229828:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aea5e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f63 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.229941:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aea65 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.229957:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aea65 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.229968:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aea65 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845043.229993:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 912867 00000004:00200000:3.0:1788845043.229998:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845043.230016:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aea65 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.230042:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aea5e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f63 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.230060:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aea65 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f63 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.230070:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.230073:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845043.230076:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:3.0:1788845043.230139:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641766784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 356us (649us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.230146:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1500 00000100:00100000:2.0:1788845043.233579:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.233623:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.233721:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641766912 00000100:00100000:3.0:1788845043.233731:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641766912 00000100:00100000:3.0:1788845043.233785:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1501 00000100:00100000:3.0:1788845043.233795:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641766912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.233816:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.233835:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287aea6c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f6a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.233872:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.233937:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660380416:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.234019:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e220c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845043.236536:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845041 -> 1788845043 00000004:00010000:2.0:1788845043.236557:0:5841:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 198 -> 262 00000100:00100000:2.0:1788845043.236584:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660380416:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.236607:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.236625:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287aea6c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f6a expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.236644:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.236647:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.236652:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287aea6c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f6a expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.236680:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:2.0:1788845043.236744:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641766912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2951us (3175us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.236753:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1501 00000100:00100000:1.0:1788845043.238448:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641767040 00000100:00100000:1.0:1788845043.238463:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641767040 00000100:00100000:1.0:1788845043.238527:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1088 00000100:00100000:1.0:1788845043.238538:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641767040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.238567:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.238585:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287aea73 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f71 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.238650:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660380544:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.238916:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00002000:00010000:1.0:1788845043.241748:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845042 -> 1788845043 00000100:00100000:1.0:1788845043.241780:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660380544:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845043.241817:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287aea73 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f71 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.241843:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd40329b/0xb2187301287aea73 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f71 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.241857:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845043.241948:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641767040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3412us (8323us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.241956:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1088 00000100:00100000:3.0:1788845043.244940:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.245048:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641767168 00000100:00100000:2.0:1788845043.245058:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641767168 00000100:00100000:2.0:1788845043.245117:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1502 00000100:00100000:2.0:1788845043.245126:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641767168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.245147:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.245167:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aea7a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f78 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.245220:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.245275:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660380672:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.245365:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.251456:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.251591:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641767296 00000100:00100000:2.0:1788845043.251602:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641767296 00000100:00100000:2.0:1788845043.251675:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1089 00000100:00100000:2.0:1788845043.251686:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641767296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.251705:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.251724:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aea81 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f7f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.251784:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660380800:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.251870:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.252576:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660380672:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.252596:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:3.0:1788845043.252613:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aea7a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f78 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.252633:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.252636:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.252641:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aea7a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f78 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.252657:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:3.0:1788845043.252718:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641767168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7592us (7787us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.252726:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1502 00000100:00100000:2.0:1788845043.260141:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660380800:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.260199:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aea81 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f7f expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.260225:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aea81 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f7f expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.260238:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:2.0:1788845043.260305:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641767296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8620us (8858us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.260315:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1089 00000100:00100000:0.0:1788845043.263587:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.263628:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.263729:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641767552 00000100:00100000:1.0:1788845043.263738:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641767552 00000100:00100000:1.0:1788845043.263796:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1090 00000100:00100000:1.0:1788845043.263807:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641767552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.263826:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.263843:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287aea88 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f8d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.263938:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660380928:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.264011:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.264989:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641767424 00000100:00100000:2.0:1788845043.264998:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641767424 00000100:00100000:2.0:1788845043.265047:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1503 00000100:00100000:2.0:1788845043.265057:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641767424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.265072:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.265089:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aea8f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f86 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.265120:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.265151:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660381056:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.265222:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.274092:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660381056:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.274122:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.274141:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aea8f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f86 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.274161:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.274164:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.274169:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aea8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f86 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.274185:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f600, rc 0) 00000100:00100000:2.0:1788845043.274263:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641767424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9207us (10684us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.274271:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1503 00000100:00100000:1.0:1788845043.274388:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660380928:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845043.274408:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287aea88 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f8d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.274426:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b142460/0xb2187301287aea88 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f8d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.274435:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:1.0:1788845043.274459:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641767552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10655us (10830us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.274466:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1090 00000100:00100000:3.0:1788845043.280293:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.280334:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.280418:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641767808 00000100:00100000:0.0:1788845043.280427:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641767808 00000100:00100000:0.0:1788845043.280481:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1091 00000100:00100000:0.0:1788845043.280493:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641767808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.280511:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.280529:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287aea96 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f9b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.280555:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641767680 00000100:00100000:2.0:1788845043.280559:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641767680 00000100:00100000:0.0:1788845043.280583:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660381184:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.280586:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1504 00000100:00100000:2.0:1788845043.280592:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641767680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.280601:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.280613:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aea9d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9f94 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.280642:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.280664:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660381312:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.280664:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008312180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.280718:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00010000:00010000:1.0:1788845043.282014:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845043.285133:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660381312:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.285162:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.285182:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aea9d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f94 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.285215:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.285219:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.285224:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aea9d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f94 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.285239:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f600, rc 0) 00000100:00100000:2.0:1788845043.285304:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641767680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4710us (5020us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.285313:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1504 00000100:00100000:3.0:1788845043.286970:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660381184:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.287005:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287aea96 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f9b expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.287027:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287aea96 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f9b expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.287039:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:3.0:1788845043.287083:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641767808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6593us (6748us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.287091:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1091 00000100:00100000:3.0:1788845043.290437:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.290485:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.290600:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641767936 00000100:00100000:1.0:1788845043.290610:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641767936 00000100:00100000:1.0:1788845043.290680:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1505 00000100:00100000:1.0:1788845043.290692:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641767936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.290713:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.290734:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287aeaa4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fa2 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.290772:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.290813:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660381440:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.290914:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.292008:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641768064 00000100:00100000:0.0:1788845043.292017:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641768064 00000100:00100000:0.0:1788845043.292068:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1092 00000100:00100000:0.0:1788845043.292077:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641768064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.292092:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.292105:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287aeaab lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fa9 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.292148:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660381568:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.292453:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.295115:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660381568:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.295156:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287aeaab lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fa9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.295182:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287aeaab lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fa9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.295196:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:3.0:1788845043.295248:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641768064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3171us (4762us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.295268:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1092 00000100:00100000:0.0:1788845043.295385:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660381440:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845043.295400:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:0.0:1788845043.295412:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287aeaa4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fa2 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.295426:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.295429:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845043.295434:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14000/0xb2187301287aeaa4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fa2 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.295445:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14000, rc 0) 00000100:00100000:0.0:1788845043.295469:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641767936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4782us (5044us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.295476:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1505 00000100:00100000:2.0:1788845043.299093:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.299137:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.299339:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641768192 00000100:00100000:3.0:1788845043.299349:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641768192 00000100:00100000:3.0:1788845043.299408:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1506 00000100:00100000:3.0:1788845043.299418:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641768192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.299440:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.299463:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aeab2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fb0 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.299505:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.299564:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660381696:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.299652:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.300216:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641768320 00000100:00100000:1.0:1788845043.300223:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641768320 00000100:00100000:1.0:1788845043.300263:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1093 00000100:00100000:1.0:1788845043.300271:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641768320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.300281:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.300293:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287aeab9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fb7 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.300333:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660381824:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.300388:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.304118:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660381696:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.304145:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:3.0:1788845043.304164:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aeab2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fb0 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.304194:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.304198:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.304203:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aeab2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fb0 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.304219:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030600, rc 0) 00000100:00100000:3.0:1788845043.304273:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641768192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4856us (5189us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.304282:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1506 00000100:00100000:1.0:1788845043.304861:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660381824:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845043.304918:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287aeab9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fb7 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.304937:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287aeab9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fb7 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.304946:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:1.0:1788845043.304982:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641768320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4712us (5844us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.304989:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1093 00000100:00100000:2.0:1788845043.308653:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.308739:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641768448 00000100:00100000:3.0:1788845043.308749:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641768448 00000100:00100000:3.0:1788845043.308812:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1507 00000100:00100000:3.0:1788845043.308823:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641768448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.308844:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.308866:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fbe expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.308934:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.308945:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.308975:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.308993:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 10 pid: 5841 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.309007:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660381952:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845043.309082:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.313612:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.313722:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660381952:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845043.313756:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.313775:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.313779:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.313783:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.313794:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:3.0:1788845043.313851:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641768448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5030us (5207us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.313860:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1507 00000100:00100000:0.0:1788845043.314343:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641768576 00000100:00100000:0.0:1788845043.314351:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641768576 00010000:00010000:0.0:1788845043.314381:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 11 pid: 5841 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.314420:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 511 00000100:00100000:0.0:1788845043.314429:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641768576:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845043.314443:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845043.314468:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 11 pid: 5841 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.314483:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 11 pid: 5841 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.314505:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aea49 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f55 expref: 11 pid: 5841 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.314519:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.314524:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.314534:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.314554:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.314589:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.314608:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660382080:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845043.314678:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.318579:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660382080:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845043.318606:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845043.318674:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641768576:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4235us (5062us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.318686:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 511 00000100:00100000:1.0:1788845043.321036:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.321129:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641768704 00000100:00100000:3.0:1788845043.321140:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641768704 00000100:00100000:3.0:1788845043.321216:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 990 00000100:00100000:3.0:1788845043.321227:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641768704:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845043.321526:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 912867 -> 912867 00000004:00010000:3.0:1788845043.321530:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.321532:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.321534:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.321536:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 262 -> 262 00000100:00100000:3.0:1788845043.321677:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641768704:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 442us (640us total) trans 4294967835 rc 0/0 00000100:00100000:3.0:1788845043.321685:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 990 00000100:00100000:2.0:1788845043.324993:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.325084:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641768832 00000100:00100000:3.0:1788845043.325096:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641768832 00000100:00100000:3.0:1788845043.325158:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1508 00000100:00100000:3.0:1788845043.325168:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641768832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.325200:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.325220:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aeac7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fc5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.325258:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.325297:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660382208:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.325755:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.327931:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660382208:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.327956:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:3.0:1788845043.327976:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aeac7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fc5 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.327998:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.328001:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.328006:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aeac7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fc5 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.328021:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030600, rc 0) 00000100:00100000:3.0:1788845043.328115:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641768832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2948us (3131us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.328123:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1508 00000100:00100000:1.0:1788845043.331024:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.331126:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641768960 00000100:00100000:0.0:1788845043.331135:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641768960 00000100:00100000:0.0:1788845043.331330:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1509 00000100:00100000:0.0:1788845043.331342:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641768960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.331363:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.331383:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aeace lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fcc expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.331418:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.331459:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660382336:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.331588:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.334079:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660382336:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845043.334106:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:0.0:1788845043.334127:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aeace lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fcc expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.334152:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.334156:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845043.334162:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aeace lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fcc expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.334177:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845043.334486:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641768960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3145us (3469us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.334494:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1509 00000100:00100000:3.0:1788845043.337744:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.337818:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641769088 00000100:00100000:0.0:1788845043.337827:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641769088 00000100:00100000:0.0:1788845043.337907:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1510 00000100:00100000:0.0:1788845043.337918:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641769088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.337938:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.337956:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287aead5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fd3 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.337993:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.338037:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660382464:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.338165:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.341170:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660382464:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845043.341335:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:0.0:1788845043.341357:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287aead5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fd3 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.341380:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.341383:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845043.341389:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287aead5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fd3 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.341403:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:0.0:1788845043.341503:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641769088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3585us (3767us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.341512:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1510 00000100:00100000:3.0:1788845043.344222:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.344291:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641769216 00000100:00100000:2.0:1788845043.344301:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641769216 00000100:00100000:2.0:1788845043.344360:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1511 00000100:00100000:2.0:1788845043.344371:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641769216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.344392:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.344413:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aeadc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fda expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.344449:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.344493:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660382592:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.344810:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.346772:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660382592:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.346798:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.346816:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aeadc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fda expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.346837:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.346841:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.346846:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aeadc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fda expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.346861:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:2.0:1788845043.346946:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641769216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2577us (2732us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.346955:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1511 00000100:00100000:1.0:1788845043.350654:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.350790:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641769472 00000100:00100000:3.0:1788845043.350801:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641769472 00000100:00100000:3.0:1788845043.350861:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1512 00000100:00100000:3.0:1788845043.350873:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641769472:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845043.350968:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845043.350996:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aea65 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f63 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.351016:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.351021:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845043.351697:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.351716:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.351725:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.351760:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 9 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.351779:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 9 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.351795:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660382720:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845043.351876:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.354516:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660382720:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845043.354555:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.356509:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.356761:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641769600 00000100:00100000:2.0:1788845043.356771:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641769600 00010000:00010000:2.0:1788845043.356809:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 10 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.356851:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 512 00000100:00100000:2.0:1788845043.356862:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641769600:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845043.356877:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845043.356921:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 10 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.356940:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 10 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.356963:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aea57 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9f5c expref: 10 pid: 5839 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.356980:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.356986:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.356996:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.357004:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.357032:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845043.357083:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641769600:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 223us (581us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845043.357091:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 512 00010000:00010000:3.0:1788845043.357516:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.357525:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845043.357653:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967836 00000004:00010000:3.0:1788845043.357712:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 912867 -> 912867 00000004:00010000:3.0:1788845043.357717:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.357720:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.357722:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.357724:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 262 -> 262 00000004:00010000:3.0:1788845043.357731:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000294c5545 reply state 00000000de8ff796 transno 4294967837 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.357740:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845043.357826:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641769472:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6954us (7181us total) trans 4294967837 rc 0/0 00000100:00100000:3.0:1788845043.357834:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1512 00000020:00080000:3.0:1788845043.357837:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845043.360933:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.360950:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.360957:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.360963:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031d3f600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845043.360968:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.360977:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.360993:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.360996:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845043.361001:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.361009:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aeae3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.361760:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.362771:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641769728 00000100:00100000:2.0:1788845043.362784:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641769728 00000100:00100000:2.0:1788845043.362904:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1094 00000100:00100000:2.0:1788845043.362916:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641769728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.362936:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.362956:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fe1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.362989:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.363000:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(17 14) 00010000:00010000:2.0:1788845043.363005:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 17) 00010000:00010000:2.0:1788845043.363010:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.363046:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.363065:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 6 pid: 6684 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.363080:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660382848:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845043.363167:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.366103:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.366173:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641769856 00000100:00100000:0.0:1788845043.366183:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641769856 00010000:00010000:0.0:1788845043.366242:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 7 pid: 6684 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.366288:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 513 00000100:00100000:0.0:1788845043.366299:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641769856:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845043.366316:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845043.366340:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 7 pid: 6684 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.366358:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 7 pid: 6684 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.366384:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 17) 00010000:00010000:0.0:1788845043.366388:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 17) 00010000:00010000:0.0:1788845043.366396:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.366406:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.366440:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.366455:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660382976:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845043.366531:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0083129c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788845043.368331:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287ae8cf lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1241088->1265663) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9e0c expref: 7 pid: 6684 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.368353:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660382848:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845043.368382:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.368400:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:2.0:1788845043.368467:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641769728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5553us (6713us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845043.368475:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1094 00000100:00100000:0.0:1788845043.372046:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660382976:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845043.372075:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845043.372131:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641769856:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5834us (6038us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.372142:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 513 00000100:00100000:3.0:1788845043.373470:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.373557:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641769984 00000100:00100000:2.0:1788845043.373566:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641769984 00002000:00010000:2.0:1788845043.373602:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (2051244->18446744073709551615) 00002000:00100000:2.0:1788845043.373608:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db9f80 with x1875739641769984 ext(2051244->18446744073709551615) 00002000:00010000:2.0:1788845043.373618:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db9f80. 00000100:00100000:2.0:1788845043.373650:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 515 00000100:00100000:2.0:1788845043.373660:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641769984:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845043.373933:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 2051244 00002000:00010000:2.0:1788845043.373939:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00002000:00010000:2.0:1788845043.373941:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845043 -> 1788845042 00002000:00010000:2.0:1788845043.373943:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00002000:00010000:2.0:1788845043.373945:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 203 -> 203 00000100:00100000:2.0:1788845043.374004:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641769984:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 345us (540us total) trans 4294967490 rc 0/0 00000100:00100000:2.0:1788845043.374011:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 515 00002000:00010000:2.0:1788845043.374014:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (2051244->18446744073709551615) 00002000:00100000:2.0:1788845043.374017:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db9f80 with x1875739641769984 ext(2051244->18446744073709551615) 00002000:00010000:2.0:1788845043.374022:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db9f80. 00000100:00100000:1.0:1788845043.382997:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.383049:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.383171:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641770240 00000100:00100000:3.0:1788845043.383181:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641770240 00002000:00100000:3.0:1788845043.383243:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918700 x1875739641770240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.383255:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918700 with x1875739641770240 00000100:00100000:3.0:1788845043.383293:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 991 00000100:00100000:3.0:1788845043.383303:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641770240:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845043.384560:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641770112 00000100:00100000:2.0:1788845043.384570:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641770112 00002000:00100000:2.0:1788845043.384606:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2052095) req@ffff949f0383e300 x1875739641770112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.384616:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383e300 with x1875739641770112 ext(1048576->2052095) 00002000:00010000:2.0:1788845043.384622:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383e300 00000100:00100000:2.0:1788845043.384652:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 516 00000100:00100000:2.0:1788845043.384661:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641770112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.396782:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641770240:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 13467us (13719us total) trans 0 rc 270336/270336 00000100:00100000:3.0:1788845043.396798:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 991 00002000:00100000:3.0:1788845043.396815:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008918700 x1875739641770240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 270336/270336 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.396826:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008918700 with x1875739641770240 00000100:00100000:2.0:1788845043.405355:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641770112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 20684us (22357us total) trans 0 rc 1003520/1003520 00000100:00100000:2.0:1788845043.405371:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 516 00002000:00100000:2.0:1788845043.405386:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2052095) req@ffff949f0383e300 x1875739641770112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 1003520/1003520 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.405398:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383e300 with x1875739641770112 ext(1048576->2052095) 00002000:00010000:2.0:1788845043.405405:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383e300 00000100:00100000:1.0:1788845043.410323:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.410533:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641770368 00000100:00100000:2.0:1788845043.410553:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641770368 00002000:00100000:2.0:1788845043.410602:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (2027520->2051243) req@ffff94a008919880 x1875739641770368/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.410613:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a008919880 with x1875739641770368 ext(2027520->2051243) 00002000:00010000:2.0:1788845043.410618:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a008919880 00000100:00100000:2.0:1788845043.410646:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 517 00000100:00100000:2.0:1788845043.410656:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641770368:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845043.415761:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641770368:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5092us (5433us total) trans 4294967491 rc 0/0 00000100:00100000:2.0:1788845043.415780:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 517 00002000:00100000:2.0:1788845043.415794:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (2027520->2051243) req@ffff94a008919880 x1875739641770368/t4294967491(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/448 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.415806:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a008919880 with x1875739641770368 ext(2027520->2051243) 00002000:00010000:2.0:1788845043.415812:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a008919880 00000100:00100000:1.0:1788845043.418848:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.418965:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641770496 00000100:00100000:3.0:1788845043.418975:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641770496 00000100:00100000:3.0:1788845043.419065:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1095 00000100:00100000:3.0:1788845043.419077:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641770496:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:2.0:1788845043.445533:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967489 is committed 00000001:00080000:2.0:1788845043.445563:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967491 is committed 00000100:00100000:1.0:1788845043.445722:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641770496:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 26642us (26876us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845043.445734:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1095 00000100:00100000:3.0:1788845043.448281:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.448449:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641770624 00000100:00100000:1.0:1788845043.448459:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641770624 00000100:00100000:1.0:1788845043.448631:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1513 00000100:00100000:1.0:1788845043.448643:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641770624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.448674:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.448696:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287aeaf1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fe8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.448780:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.448794:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.448805:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845043.448830:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2051244 00000004:00200000:1.0:1788845043.448834:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845043.448852:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.448877:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15a00/0xb2187301287aeaf1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.448934:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.448944:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845043.448948:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845043.448951:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14200, rc 0) 00000100:00100000:1.0:1788845043.449004:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641770624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 363us (732us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845043.449015:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1513 00000100:00100000:3.0:1788845043.451322:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.453061:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.453096:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641770880 00000100:00100000:3.0:1788845043.453106:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641770880 00000100:00100000:3.0:1788845043.453158:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1096 00000100:00100000:3.0:1788845043.453169:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641770880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.453188:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.453223:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287aeaff lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ff6 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.453283:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660383104:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.453369:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22480 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.455006:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641770752 00000100:00100000:2.0:1788845043.455018:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641770752 00000100:00100000:2.0:1788845043.455069:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1514 00000100:00100000:2.0:1788845043.455077:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641770752:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.455093:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.455113:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aeb06 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9fef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.455145:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.455179:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660383232:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.455255:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00002000:00010000:3.0:1788845043.455569:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845042 -> 1788845043 00000100:00100000:3.0:1788845043.455589:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660383104:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.455614:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287aeaff lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ff6 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.455639:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287aeaff lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ff6 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.455651:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:3.0:1788845043.455688:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641770880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2522us (2635us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.455696:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1096 00000100:00100000:3.0:1788845043.458140:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660383232:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.458166:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:3.0:1788845043.458195:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aeb06 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fef expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.458214:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.458217:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.458222:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aeb06 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fef expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.458235:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:3.0:1788845043.458402:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641770752:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3323us (7086us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.458412:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1514 00000100:00100000:3.0:1788845043.460814:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.460875:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.460931:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641771008 00000100:00100000:3.0:1788845043.460940:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641771008 00000100:00100000:3.0:1788845043.460991:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1515 00000100:00100000:3.0:1788845043.461000:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641771008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.461020:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.461039:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287aeb0d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109a9ffd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.461075:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.461114:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01dbbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660383360:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.461197:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22480 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.461730:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641771136 00000100:00100000:2.0:1788845043.461735:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641771136 00000100:00100000:2.0:1788845043.461781:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1097 00000100:00100000:2.0:1788845043.461789:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641771136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.461799:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.461812:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000518aaa22/0xb2187301287aeb14 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa004 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.461853:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660383488:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.461926:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.464530:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660383488:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.464577:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000518aaa22/0xb2187301287aeb14 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa004 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.464601:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000518aaa22/0xb2187301287aeb14 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa004 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.464614:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f600, rc 0) 00000100:00100000:3.0:1788845043.464661:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641771136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2873us (3856us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.464678:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1097 00000100:00100000:2.0:1788845043.465478:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01dbbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660383360:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.465498:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.465514:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287aeb0d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ffd expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.465531:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.465534:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.465550:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02032c00/0xb2187301287aeb0d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9ffd expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.465564:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032c00, rc 0) 00000100:00100000:2.0:1788845043.465610:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641771008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4611us (4735us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.465617:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1515 00000100:00100000:3.0:1788845043.470423:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.470474:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.470558:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641771392 00000100:00100000:2.0:1788845043.470570:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641771392 00000100:00100000:2.0:1788845043.470639:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1098 00000100:00100000:2.0:1788845043.470650:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641771392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.470677:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.470697:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aeb1b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa012 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.470765:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660383616:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.470851:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.471438:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641771264 00000100:00100000:0.0:1788845043.471444:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641771264 00000100:00100000:0.0:1788845043.471485:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1516 00000100:00100000:0.0:1788845043.471492:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641771264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.471505:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.471523:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287aeb22 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa00b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.471575:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.471611:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660383744:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.471744:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.475113:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660383744:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845043.475141:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:0.0:1788845043.475161:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287aeb22 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa00b expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.475201:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.475205:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845043.475210:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287aeb22 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa00b expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.475225:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845043.475290:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641771264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3797us (4875us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.475299:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1516 00000100:00100000:2.0:1788845043.477040:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660383616:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.477074:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aeb1b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa012 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.477095:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aeb1b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa012 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.477107:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f800, rc 0) 00000100:00100000:2.0:1788845043.477159:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641771392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6511us (6683us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.477167:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1098 00000100:00100000:1.0:1788845043.484789:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.484834:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.484955:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641771520 00000100:00100000:2.0:1788845043.484965:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641771520 00000100:00100000:2.0:1788845043.485031:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1517 00000100:00100000:2.0:1788845043.485042:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641771520:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.485061:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.485081:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aeb29 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa019 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.485118:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.485165:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660383872:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.485252:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.485422:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641771648 00000100:00100000:3.0:1788845043.485427:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641771648 00000100:00100000:3.0:1788845043.485465:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1099 00000100:00100000:3.0:1788845043.485473:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641771648:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.485481:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.485491:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287aeb30 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa020 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.485526:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660384000:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.485553:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.488451:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660384000:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845043.488506:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287aeb30 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa020 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.488529:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffb1c6aa/0xb2187301287aeb30 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa020 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.488553:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02032400, rc 0) 00000100:00100000:1.0:1788845043.488618:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641771648:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3145us (3781us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.488625:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1099 00000100:00100000:2.0:1788845043.488864:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660383872:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.488922:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.488940:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aeb29 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa019 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.488956:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.488959:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.488963:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aeb29 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa019 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.488976:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:2.0:1788845043.489004:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641771520:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3965us (4225us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.489011:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1517 00000100:00100000:3.0:1788845043.492364:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.492523:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641771776 00000100:00100000:1.0:1788845043.492533:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641771776 00000100:00100000:1.0:1788845043.492609:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1518 00000100:00100000:1.0:1788845043.492620:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641771776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.492641:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.492660:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287aeb37 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa027 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.492764:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287aeb3e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.492785:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287aeb3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.492798:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287aeb3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845043.492828:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2051244 00000004:00200000:1.0:1788845043.492832:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845043.492849:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287aeb3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.492876:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287aeb37 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa027 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.492920:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287aeb3e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa027 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.492929:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845043.492932:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845043.492936:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14e00, rc 0) 00000100:00100000:1.0:1788845043.493007:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641771776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 389us (653us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845043.493015:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1518 00000100:00100000:3.0:1788845043.498845:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.498929:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.499018:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641772032 00000100:00100000:0.0:1788845043.499028:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641772032 00000100:00100000:0.0:1788845043.499086:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1100 00000100:00100000:0.0:1788845043.499097:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641772032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.499116:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.499135:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aeb45 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa035 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.499205:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660384128:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.499286:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.499874:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641771904 00000100:00100000:1.0:1788845043.499908:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641771904 00000100:00100000:1.0:1788845043.499954:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1519 00000100:00100000:1.0:1788845043.499962:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641771904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.499976:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.499992:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aeb4c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa02e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.500021:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.500049:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660384256:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.500098:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.503190:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660384256:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845043.503215:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:1.0:1788845043.503236:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aeb4c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa02e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.503270:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845043.503273:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845043.503278:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14600/0xb2187301287aeb4c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa02e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.503293:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14600, rc 0) 00000100:00100000:1.0:1788845043.503355:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641771904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3393us (4517us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.503363:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1519 00000100:00100000:0.0:1788845043.503837:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660384128:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845043.503860:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aeb45 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa035 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.503877:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aeb45 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa035 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.503931:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845043.503970:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641772032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4875us (5041us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.503978:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1100 00000100:00100000:3.0:1788845043.508990:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.509027:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.509163:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641772160 00000100:00100000:1.0:1788845043.509174:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641772160 00000100:00100000:1.0:1788845043.509229:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1520 00000100:00100000:1.0:1788845043.509241:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641772160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.509274:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.509297:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287aeb53 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa03c expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.509336:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.509379:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660384384:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.509454:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.509705:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641772288 00000100:00100000:2.0:1788845043.509710:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641772288 00000100:00100000:2.0:1788845043.509753:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1101 00000100:00100000:2.0:1788845043.509761:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641772288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.509771:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.509782:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aeb5a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa043 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.509817:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660384512:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.509845:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.512760:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660384512:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.512797:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aeb5a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa043 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.512822:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aeb5a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa043 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.512835:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:2.0:1788845043.512920:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641772288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3158us (3890us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.512928:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1101 00000100:00100000:1.0:1788845043.515112:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660384384:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845043.515138:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:1.0:1788845043.515156:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287aeb53 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa03c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.515175:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845043.515178:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845043.515194:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14c00/0xb2187301287aeb53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa03c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.515209:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14c00, rc 0) 00000100:00100000:1.0:1788845043.515330:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641772160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6017us (6275us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.515339:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1520 00000100:00100000:3.0:1788845043.520206:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.520314:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641772416 00000100:00100000:2.0:1788845043.520324:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641772416 00002000:00100000:2.0:1788845043.520373:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4a300 x1875739641772416/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.520386:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4a300 with x1875739641772416 00000100:00100000:2.0:1788845043.520423:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 992 00000100:00100000:2.0:1788845043.520432:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641772416:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845043.520716:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641772416:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 283us (518us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845043.520725:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 992 00002000:00100000:2.0:1788845043.520733:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4a300 x1875739641772416/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.520746:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4a300 with x1875739641772416 00000100:00100000:0.0:1788845043.525195:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.525306:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641772544 00000100:00100000:1.0:1788845043.525317:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641772544 00002000:00100000:1.0:1788845043.525365:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6d80 x1875739641772544/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.525376:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6d80 with x1875739641772544 00000100:00100000:1.0:1788845043.525413:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 993 00000100:00100000:1.0:1788845043.525423:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641772544:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845043.525689:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641772544:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (513us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845043.525697:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 993 00002000:00100000:1.0:1788845043.525705:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6d80 x1875739641772544/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.525714:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6d80 with x1875739641772544 00000100:00100000:3.0:1788845043.529611:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.529716:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641772672 00000100:00100000:2.0:1788845043.529726:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641772672 00002000:00100000:2.0:1788845043.529774:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48000 x1875739641772672/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 8680/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.529785:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48000 with x1875739641772672 00000100:00100000:2.0:1788845043.529827:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 994 00000100:00100000:2.0:1788845043.529838:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641772672:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845043.530237:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 912867 -> 912867 00000004:00010000:2.0:1788845043.530241:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00000004:00010000:2.0:1788845043.530243:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:2.0:1788845043.530246:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00000004:00010000:2.0:1788845043.530248:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 262 -> 262 00000100:00100000:2.0:1788845043.530308:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641772672:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 471us (704us total) trans 4294967838 rc 0/0 00000100:00100000:2.0:1788845043.530315:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 994 00002000:00100000:2.0:1788845043.530323:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48000 x1875739641772672/t4294967838(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 8680/448 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.530331:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48000 with x1875739641772672 00000100:00100000:0.0:1788845043.532924:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.533060:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641772800 00000100:00100000:2.0:1788845043.533070:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641772800 00000100:00100000:2.0:1788845043.533162:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1521 00000100:00100000:2.0:1788845043.533173:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641772800:12345-192.168.201.29@tcp:16:fsx.0 00000100:00100000:2.0:1788845043.560224:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641772800:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 27042us (27339us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845043.560241:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1521 00000001:00080000:0.0:1788845043.560387:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967838 is committed 00000100:00100000:0.0:1788845043.563828:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.563912:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641773056 00000100:00100000:3.0:1788845043.563922:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641773056 00000100:00100000:3.0:1788845043.563985:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1522 00000100:00100000:3.0:1788845043.563996:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641773056:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845043.564047:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845043.564072:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14e00/0xb2187301287aeb3e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa027 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.564091:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.564095:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845043.564369:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.564386:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.564394:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.564425:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 9 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.564443:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 9 pid: 11036 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.564457:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660384640:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845043.564823:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.568439:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.568776:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641773184 00000100:00100000:2.0:1788845043.568784:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641773184 00010000:00010000:2.0:1788845043.568820:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 10 pid: 11036 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.568860:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 514 00000100:00100000:2.0:1788845043.568869:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641773184:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845043.568897:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845043.568904:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 10 pid: 11036 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.568922:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 10 pid: 11036 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.568944:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.568950:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.568960:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.568968:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.568985:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845043.569037:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641773184:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 169us (606us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845043.569048:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 514 00010000:00010000:3.0:1788845043.569366:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d14200/0xb2187301287aeaf8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe8 expref: 9 pid: 11036 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.569385:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660384640:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845043.569408:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.569417:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.569423:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845043.569638:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967839 00000004:00010000:3.0:1788845043.569697:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 912867 -> 912867 00000004:00010000:3.0:1788845043.569701:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.569704:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.569706:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845043.569708:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 262 -> 326 00000004:00010000:3.0:1788845043.569715:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f7587e85 reply state 000000003a2881a3 transno 4294967840 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.569723:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845043.569807:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641773056:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5812us (5987us total) trans 4294967840 rc 0/0 00000100:00100000:3.0:1788845043.569814:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1522 00000020:00080000:3.0:1788845043.569818:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845043.574413:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.574494:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641773312 00000100:00100000:1.0:1788845043.574503:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641773312 00002000:00010000:1.0:1788845043.574552:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (2081862->18446744073709551615) 00002000:00100000:1.0:1788845043.574558:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8d180 with x1875739641773312 ext(2081862->18446744073709551615) 00002000:00010000:1.0:1788845043.574569:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8d180. 00000100:00100000:1.0:1788845043.574602:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 518 00000100:00100000:1.0:1788845043.574612:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641773312:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845043.574923:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2051244 -> 2081862 00002000:00010000:1.0:1788845043.574927:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00002000:00010000:1.0:1788845043.574930:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00002000:00010000:1.0:1788845043.574932:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00002000:00010000:1.0:1788845043.574935:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 203 -> 57 00000100:00100000:1.0:1788845043.574995:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641773312:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 383us (591us total) trans 4294967492 rc 0/0 00000100:00100000:1.0:1788845043.575003:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 518 00002000:00010000:1.0:1788845043.575006:0:6689:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (2081862->18446744073709551615) 00002000:00100000:1.0:1788845043.575009:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8d180 with x1875739641773312 ext(2081862->18446744073709551615) 00002000:00010000:1.0:1788845043.575013:0:6689:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8d180. 00010000:00010000:2.0:1788845043.575222:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.575232:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.575239:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.575244:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02033400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845043.575248:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.575265:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.575281:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.575283:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845043.575287:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.575293:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033400/0xb2187301287aeb61 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.579459:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.579558:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641773440 00000100:00100000:0.0:1788845043.579569:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641773440 00000100:00100000:0.0:1788845043.579795:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1523 00000100:00100000:0.0:1788845043.579806:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641773440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.579830:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.579850:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287aeb68 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa04a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.579966:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.579981:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.579993:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845043.580022:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2081862 00000004:00200000:0.0:1788845043.580026:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845043.580045:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.580069:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287aeb68 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.580085:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.580093:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.580096:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845043.580099:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845043.580218:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641773440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 414us (767us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.580224:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1523 00000100:00100000:0.0:1788845043.583845:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.583969:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641773568 00000100:00100000:1.0:1788845043.583979:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641773568 00000100:00100000:1.0:1788845043.584040:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1524 00000100:00100000:1.0:1788845043.584052:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641773568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.584072:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.584094:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287aeb76 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa051 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.584131:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.584170:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f777100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660384768:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.584267:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.587024:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.587348:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641773696 00000100:00100000:2.0:1788845043.587358:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641773696 00000100:00100000:2.0:1788845043.587416:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1102 00000100:00100000:2.0:1788845043.587426:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641773696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.587444:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.587463:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aeb7d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa058 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.587523:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660384896:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.587616:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.588068:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f777100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660384768:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845043.588089:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:1.0:1788845043.588107:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287aeb76 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa051 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.588127:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845043.588131:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845043.588136:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f01d15800/0xb2187301287aeb76 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa051 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.588151:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15800, rc 0) 00000100:00100000:1.0:1788845043.588207:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641773568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4156us (4580us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.588214:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1524 00000100:00100000:2.0:1788845043.593591:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660384896:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.593751:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aeb7d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa058 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.593779:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aeb7d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa058 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.593792:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:2.0:1788845043.593854:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641773696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6429us (6840us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.593861:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1102 00000100:00100000:0.0:1788845043.597170:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.597319:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641773824 00000100:00100000:2.0:1788845043.597328:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641773824 00000100:00100000:2.0:1788845043.597372:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1525 00000100:00100000:2.0:1788845043.597380:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641773824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.597400:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.597418:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aeb84 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa05f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.597458:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.597502:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660385024:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.597586:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.599980:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.600144:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660385024:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.600169:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.600200:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aeb84 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa05f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.600221:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.600224:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.600229:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aeb84 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa05f expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.600246:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f800, rc 0) 00000100:00100000:2.0:1788845043.600298:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641773824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2917us (3138us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.600306:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1525 00000100:00100000:3.0:1788845043.601220:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641773952 00000100:00100000:3.0:1788845043.601231:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641773952 00000100:00100000:3.0:1788845043.601289:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1103 00000100:00100000:3.0:1788845043.601299:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641773952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.601316:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.601331:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287aeb8b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa066 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.601385:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660385152:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.601767:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.604649:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660385152:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845043.604700:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287aeb8b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa066 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.604732:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287aeb8b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa066 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.604746:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033a00, rc 0) 00000100:00100000:3.0:1788845043.604809:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641773952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3509us (4838us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.604817:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1103 00000100:00100000:1.0:1788845043.610986:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.611025:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.611102:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641774208 00000100:00100000:0.0:1788845043.611112:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641774208 00000100:00100000:0.0:1788845043.611171:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1104 00000100:00100000:0.0:1788845043.611192:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641774208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.611210:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.611228:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287aeb92 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa074 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.611286:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660385280:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.611370:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.611995:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641774080 00000100:00100000:3.0:1788845043.612001:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641774080 00000100:00100000:3.0:1788845043.612050:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1526 00000100:00100000:3.0:1788845043.612059:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641774080:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.612071:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.612085:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033a00/0xb2187301287aeb99 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa06d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.612116:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.612143:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660385408:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.612200:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845043.615709:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660385408:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845043.615737:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:3.0:1788845043.615758:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033a00/0xb2187301287aeb99 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa06d expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.615785:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845043.615788:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845043.615793:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02033a00/0xb2187301287aeb99 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa06d expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.615810:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033a00, rc 0) 00000100:00100000:3.0:1788845043.615868:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641774080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3810us (4891us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845043.615878:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1526 00000100:00100000:0.0:1788845043.616682:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660385280:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845043.616708:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287aeb92 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa074 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.616729:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287aeb92 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa074 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.616741:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845043.616790:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641774208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5610us (5763us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.616797:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1104 00000100:00100000:1.0:1788845043.621940:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.622038:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641774336 00000100:00100000:0.0:1788845043.622047:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641774336 00000100:00100000:0.0:1788845043.622111:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1527 00000100:00100000:0.0:1788845043.622121:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641774336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.622141:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.622161:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aeba0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa07b expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.622212:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.622251:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660385536:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.622343:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.624642:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.625034:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660385536:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.625057:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.625074:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aeba0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa07b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.625093:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.625095:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.625100:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aeba0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa07b expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.625118:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:2.0:1788845043.625177:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641774336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3057us (3247us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.625195:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1527 00000100:00100000:0.0:1788845043.625350:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641774464 00000100:00100000:0.0:1788845043.625356:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641774464 00000100:00100000:0.0:1788845043.625406:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1105 00000100:00100000:0.0:1788845043.625416:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641774464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.625431:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.625448:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aeba7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa082 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.625498:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660385664:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.625549:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.628799:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660385664:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845043.628836:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aeba7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa082 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.628858:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aeba7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa082 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.628870:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845043.628960:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641774464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3545us (4336us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.628968:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1105 00000100:00100000:1.0:1788845043.632865:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.633005:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641774592 00000100:00100000:2.0:1788845043.633014:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641774592 00000100:00100000:2.0:1788845043.633063:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1106 00000100:00100000:2.0:1788845043.633072:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641774592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.633090:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.633107:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa089 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.633138:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.633147:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 17) 00010000:00010000:2.0:1788845043.633151:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 17) 00010000:00010000:2.0:1788845043.633156:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1216512->1277951] (req 1216512->1277951) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.633191:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.633211:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 6 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.633230:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660385792:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845043.633311:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.637157:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.637257:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641774720 00000100:00100000:3.0:1788845043.637266:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641774720 00002000:00100000:3.0:1788845043.637320:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1236992->1286143) req@ffff94a03568b800 x1875739641774720/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.637331:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03568b800 with x1875739641774720 ext(1236992->1286143) 00010000:00010000:3.0:1788845043.637341:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 7 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.637363:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 7 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845043.637371:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03568b800 00000100:00100000:3.0:1788845043.637405:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 31 00000100:00100000:3.0:1788845043.637416:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03568b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641774720:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845043.638585:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660385792:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845043.638619:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1216512->1277951] (req 1216512->1277951) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.638761:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f800, rc 0) 00000100:00100000:2.0:1788845043.638816:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641774592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5744us (5959us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845043.638824:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1106 00000100:00100000:0.0:1788845043.647149:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03568b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641774720:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 9725us (9991us total) trans 4294967493 rc 0/0 00000100:00100000:0.0:1788845043.647164:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 31 00002000:00100000:0.0:1788845043.647188:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1236992->1286143) req@ffff94a03568b800 x1875739641774720/t4294967493(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/448 e 0 to 0 dl 1788845054 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.647208:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03568b800 with x1875739641774720 ext(1236992->1286143) 00010000:00010000:0.0:1788845043.647219:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 9 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.647245:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 9 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845043.647253:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03568b800 00000100:00100000:3.0:1788845043.651145:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.651347:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641774848 00000100:00100000:0.0:1788845043.651356:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641774848 00010000:00010000:0.0:1788845043.651395:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 9 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.651492:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 515 00000100:00100000:0.0:1788845043.651503:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641774848:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845043.651519:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845043.651552:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 9 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.651570:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 9 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.651593:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aeaea lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fe1 expref: 9 pid: 6682 timeout: 532 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.651616:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 17) 00010000:00010000:0.0:1788845043.651621:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 17) 00010000:00010000:0.0:1788845043.651628:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.651773:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.651811:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.651824:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660385920:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845043.651939:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.655489:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.655681:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641774976 00000100:00100000:3.0:1788845043.655691:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641774976 00002000:00100000:3.0:1788845043.655741:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1220607) req@ffff94a03f6c0000 x1875739641774976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.655752:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c0000 with x1875739641774976 ext(1216512->1220607) 00002000:00010000:3.0:1788845043.655757:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c0000 00000100:00100000:3.0:1788845043.655794:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 519 00000100:00100000:3.0:1788845043.655804:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641774976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.656056:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641774976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (578us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.656064:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 519 00002000:00100000:3.0:1788845043.656071:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1220607) req@ffff94a03f6c0000 x1875739641774976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.656079:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c0000 with x1875739641774976 ext(1216512->1220607) 00002000:00010000:3.0:1788845043.656081:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c0000 00000100:00100000:0.0:1788845043.660542:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660385920:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845043.660571:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845043.660819:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641774848:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9316us (9682us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.660832:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 515 00000100:00100000:1.0:1788845043.661081:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.662778:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641775104 00000100:00100000:0.0:1788845043.662794:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641775104 00002000:00100000:0.0:1788845043.662852:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a03f6c2d80 x1875739641775104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.662863:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c2d80 with x1875739641775104 ext(1273856->1277951) 00002000:00010000:0.0:1788845043.662868:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c2d80 00000100:00100000:3.0:1788845043.663015:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 520 00000100:00100000:3.0:1788845043.663029:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641775104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.663474:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641775104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 447us (2393us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.663482:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 520 00002000:00100000:3.0:1788845043.663491:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a03f6c2d80 x1875739641775104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.663499:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c2d80 with x1875739641775104 ext(1273856->1277951) 00002000:00010000:3.0:1788845043.663502:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c2d80 00000100:00100000:3.0:1788845043.667325:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.667770:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641775232 00000100:00100000:0.0:1788845043.667780:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641775232 00000100:00100000:0.0:1788845043.667837:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1528 00000100:00100000:0.0:1788845043.667848:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641775232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.667871:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.667929:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aebb5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa090 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.668021:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287aebbc lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.668036:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287aebbc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.668048:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287aebbc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845043.668076:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2081862 00000004:00200000:0.0:1788845043.668081:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845043.668099:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287aebbc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.668125:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aebb5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa090 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.668143:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287aebbc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa090 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.668152:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.668155:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845043.668159:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845043.668213:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641775232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 366us (896us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.668219:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1528 00000100:00100000:3.0:1788845043.671845:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.671989:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641775360 00000100:00100000:0.0:1788845043.671999:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641775360 00000100:00100000:0.0:1788845043.672047:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1107 00000100:00100000:0.0:1788845043.672057:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641775360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.672077:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.672094:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aebc3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa097 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.672163:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660386048:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.672268:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.676429:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660386048:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845043.676470:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aebc3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa097 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.676495:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aebc3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa097 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.676507:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845043.676579:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641775360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4523us (4744us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.676589:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1107 00000100:00100000:3.0:1788845043.679079:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.679213:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641775488 00000100:00100000:0.0:1788845043.679223:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641775488 00000100:00100000:0.0:1788845043.679517:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1108 00000100:00100000:0.0:1788845043.679529:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641775488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.679557:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.679576:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aebca lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa09e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.679646:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660386176:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.679747:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.682783:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660386176:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845043.682825:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aebca lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa09e expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.682849:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aebca lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa09e expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.682861:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845043.682944:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641775488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3416us (3873us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.682953:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1108 00000100:00100000:1.0:1788845043.688091:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.688197:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641775616 00000100:00100000:2.0:1788845043.688207:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641775616 00000100:00100000:1.0:1788845043.688325:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1109 00000100:00100000:1.0:1788845043.688337:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641775616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.688356:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.688375:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287aebd1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0a5 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.688438:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660386304:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.688525:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.690914:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660386304:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.690955:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287aebd1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0a5 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.690981:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abaa71b3/0xb2187301287aebd1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0a5 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.690993:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d15c00, rc 0) 00000100:00100000:2.0:1788845043.691046:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641775616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2712us (2967us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.691054:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1109 00000100:00100000:3.0:1788845043.694427:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.694515:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641775744 00000100:00100000:2.0:1788845043.694525:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641775744 00000100:00100000:2.0:1788845043.694597:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1110 00000100:00100000:2.0:1788845043.694608:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641775744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.694627:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.694655:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aebd8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0ac expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.694729:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660386432:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.694816:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.697858:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660386432:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.697910:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aebd8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0ac expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.697935:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003329fec9/0xb2187301287aebd8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0ac expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.697947:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:2.0:1788845043.698014:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641775744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3408us (3597us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.698022:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1110 00000100:00100000:3.0:1788845043.701586:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.701777:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641775872 00000100:00100000:2.0:1788845043.701787:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641775872 00000100:00100000:2.0:1788845043.701846:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1111 00000100:00100000:2.0:1788845043.701858:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641775872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.701880:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.701922:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aebdf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0b3 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.701980:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660386560:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.702066:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.704815:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660386560:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.704853:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aebdf lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0b3 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.704876:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aebdf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0b3 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.704918:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:2.0:1788845043.704976:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641775872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3120us (3399us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.704984:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1111 00000100:00100000:0.0:1788845043.708219:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.710370:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641776000 00000100:00100000:3.0:1788845043.710387:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641776000 00000100:00100000:3.0:1788845043.710467:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1112 00000100:00100000:3.0:1788845043.710478:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641776000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845043.710500:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845043.710520:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287aebe6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0ba expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.710595:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660386688:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845043.710876:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.715540:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010481500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660386688:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845043.715583:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287aebe6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0ba expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.715616:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3213fe3/0xb2187301287aebe6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0ba expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.715631:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02033a00, rc 0) 00000100:00100000:0.0:1788845043.715704:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641776000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5227us (7502us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.715713:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1112 00000100:00100000:1.0:1788845043.718724:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.719434:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641776128 00000100:00100000:2.0:1788845043.719808:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641776128 00000100:00100000:2.0:1788845043.719875:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1529 00000100:00100000:2.0:1788845043.719902:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641776128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.719924:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.719946:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aebed lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0c1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.719989:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.720033:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660386816:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.720204:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.723605:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660386816:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.723716:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.723740:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aebed lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0c1 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.723762:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.723765:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.723771:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aebed lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0c1 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.723785:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:2.0:1788845043.723939:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641776128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4054us (5223us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.723948:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1529 00000100:00100000:2.0:1788845043.727128:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.727446:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641776256 00000100:00100000:0.0:1788845043.727457:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641776256 00000100:00100000:0.0:1788845043.727622:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1530 00000100:00100000:0.0:1788845043.727859:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641776256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.727920:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.727947:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aebf4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0c8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.727986:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.728025:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660386944:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.728103:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.731108:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660386944:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845043.731135:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:0.0:1788845043.731155:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aebf4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0c8 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.731190:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.731194:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845043.731199:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aebf4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0c8 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.731216:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845043.731295:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641776256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3662us (4176us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.731303:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1530 00000100:00100000:1.0:1788845043.738513:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.739417:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641776384 00000100:00100000:0.0:1788845043.739431:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641776384 00002000:00100000:0.0:1788845043.739489:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1900544->1904639) req@ffff94a03f6c1880 x1875739641776384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.739500:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c1880 with x1875739641776384 ext(1900544->1904639) 00002000:00010000:0.0:1788845043.739506:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c1880 00000100:00100000:0.0:1788845043.739859:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 521 00000100:00100000:0.0:1788845043.739871:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641776384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845043.740142:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641776384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 273us (1642us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845043.740150:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 521 00002000:00100000:0.0:1788845043.740158:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1900544->1904639) req@ffff94a03f6c1880 x1875739641776384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.740167:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c1880 with x1875739641776384 ext(1900544->1904639) 00002000:00010000:0.0:1788845043.740171:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c1880 00000100:00100000:3.0:1788845043.743041:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.744425:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641776512 00000100:00100000:2.0:1788845043.744440:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641776512 00002000:00100000:2.0:1788845043.744502:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1904640->1908735) req@ffff94a010482300 x1875739641776512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.744515:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010482300 with x1875739641776512 ext(1904640->1908735) 00002000:00010000:2.0:1788845043.744521:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010482300 00000100:00100000:2.0:1788845043.744564:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 522 00000100:00100000:2.0:1788845043.744573:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641776512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845043.744832:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2367:x1875739641776512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 258us (1802us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845043.744839:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 522 00002000:00100000:2.0:1788845043.744847:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1904640->1908735) req@ffff94a010482300 x1875739641776512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.744856:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010482300 with x1875739641776512 ext(1904640->1908735) 00002000:00010000:2.0:1788845043.744859:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010482300 00000100:00100000:2.0:1788845043.747978:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.748054:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641776640 00000100:00100000:3.0:1788845043.748064:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641776640 00002000:00100000:3.0:1788845043.748108:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1908736->1912831) req@ffff94a009687b80 x1875739641776640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.748117:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009687b80 with x1875739641776640 ext(1908736->1912831) 00002000:00010000:3.0:1788845043.748122:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009687b80 00000100:00100000:3.0:1788845043.748141:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 523 00000100:00100000:3.0:1788845043.748150:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641776640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.748442:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641776640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 292us (471us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.748453:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 523 00002000:00100000:3.0:1788845043.748460:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1908736->1912831) req@ffff94a009687b80 x1875739641776640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.748467:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009687b80 with x1875739641776640 ext(1908736->1912831) 00002000:00010000:3.0:1788845043.748470:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009687b80 00000100:00100000:2.0:1788845043.752119:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.752285:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641776768 00000100:00100000:0.0:1788845043.752296:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641776768 00002000:00100000:0.0:1788845043.752349:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1912832->1916927) req@ffff94a009685500 x1875739641776768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.752361:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009685500 with x1875739641776768 ext(1912832->1916927) 00002000:00010000:0.0:1788845043.752367:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009685500 00000100:00100000:0.0:1788845043.752439:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 524 00000100:00100000:0.0:1788845043.752450:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641776768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845043.752814:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641776768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 362us (703us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845043.752823:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 524 00002000:00100000:0.0:1788845043.752830:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1912832->1916927) req@ffff94a009685500 x1875739641776768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.752838:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009685500 with x1875739641776768 ext(1912832->1916927) 00002000:00010000:0.0:1788845043.752841:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009685500 00000100:00100000:2.0:1788845043.756857:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.757009:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641776896 00000100:00100000:0.0:1788845043.757019:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641776896 00002000:00100000:2.0:1788845043.757143:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1916928->1921023) req@ffff94a009686680 x1875739641776896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.757155:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009686680 with x1875739641776896 ext(1916928->1921023) 00002000:00010000:2.0:1788845043.757162:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009686680 00000100:00100000:2.0:1788845043.757313:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 525 00000100:00100000:2.0:1788845043.757327:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641776896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845043.757802:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641776896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 477us (952us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845043.757810:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 525 00002000:00100000:2.0:1788845043.757818:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1916928->1921023) req@ffff94a009686680 x1875739641776896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.757833:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009686680 with x1875739641776896 ext(1916928->1921023) 00002000:00010000:2.0:1788845043.757836:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009686680 00000100:00100000:2.0:1788845043.760992:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.761088:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641777024 00000100:00100000:3.0:1788845043.761098:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641777024 00002000:00100000:3.0:1788845043.761151:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1921024->1925119) req@ffff94a009687480 x1875739641777024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.761162:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009687480 with x1875739641777024 ext(1921024->1925119) 00002000:00010000:3.0:1788845043.761168:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009687480 00000100:00100000:3.0:1788845043.761210:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 526 00000100:00100000:3.0:1788845043.761221:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641777024:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.761515:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641777024:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 294us (532us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.761525:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 526 00002000:00100000:3.0:1788845043.761534:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1921024->1925119) req@ffff94a009687480 x1875739641777024/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.761554:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009687480 with x1875739641777024 ext(1921024->1925119) 00002000:00010000:3.0:1788845043.761559:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009687480 00000100:00100000:3.0:1788845043.768933:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.769434:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641777152 00000100:00100000:1.0:1788845043.769468:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641777152 00002000:00100000:1.0:1788845043.769521:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1925120->1929215) req@ffff94a010483100 x1875739641777152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.769550:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010483100 with x1875739641777152 ext(1925120->1929215) 00002000:00010000:1.0:1788845043.769556:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010483100 00000100:00100000:1.0:1788845043.769587:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 527 00000100:00100000:1.0:1788845043.769596:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641777152:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845043.770358:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641777152:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 759us (1472us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845043.770367:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 527 00002000:00100000:1.0:1788845043.770375:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1925120->1929215) req@ffff94a010483100 x1875739641777152/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.770383:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010483100 with x1875739641777152 ext(1925120->1929215) 00002000:00010000:1.0:1788845043.770386:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010483100 00000100:00100000:2.0:1788845043.774594:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.774768:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641777280 00000100:00100000:3.0:1788845043.774779:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641777280 00002000:00100000:3.0:1788845043.774833:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1929216->1933311) req@ffff94a009686300 x1875739641777280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.774844:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009686300 with x1875739641777280 ext(1929216->1933311) 00002000:00010000:3.0:1788845043.774850:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009686300 00000100:00100000:3.0:1788845043.774968:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 528 00000100:00100000:3.0:1788845043.774979:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641777280:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.775370:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641777280:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 391us (844us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.775379:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 528 00002000:00100000:3.0:1788845043.775386:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1929216->1933311) req@ffff94a009686300 x1875739641777280/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.775395:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009686300 with x1875739641777280 ext(1929216->1933311) 00002000:00010000:3.0:1788845043.775399:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009686300 00000100:00100000:1.0:1788845043.778483:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.778736:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641777408 00000100:00100000:2.0:1788845043.778746:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641777408 00002000:00100000:2.0:1788845043.778795:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1933312->1937407) req@ffff94a03f6c1c00 x1875739641777408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.778806:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c1c00 with x1875739641777408 ext(1933312->1937407) 00002000:00010000:2.0:1788845043.778812:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c1c00 00000100:00100000:2.0:1788845043.778845:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 529 00000100:00100000:2.0:1788845043.778854:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641777408:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845043.779250:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641777408:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 395us (775us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845043.779258:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 529 00002000:00100000:2.0:1788845043.779265:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1933312->1937407) req@ffff94a03f6c1c00 x1875739641777408/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.779273:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c1c00 with x1875739641777408 ext(1933312->1937407) 00002000:00010000:2.0:1788845043.779277:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c1c00 00000100:00100000:2.0:1788845043.783489:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.783575:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641777536 00000100:00100000:0.0:1788845043.783585:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641777536 00002000:00100000:0.0:1788845043.783647:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1937408->1941503) req@ffff94a009685f80 x1875739641777536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.783666:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009685f80 with x1875739641777536 ext(1937408->1941503) 00002000:00010000:0.0:1788845043.783672:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009685f80 00000100:00100000:0.0:1788845043.783703:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 530 00000100:00100000:0.0:1788845043.783713:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641777536:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845043.784004:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641777536:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 292us (525us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845043.784013:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 530 00002000:00100000:0.0:1788845043.784020:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1937408->1941503) req@ffff94a009685f80 x1875739641777536/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.784027:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009685f80 with x1875739641777536 ext(1937408->1941503) 00002000:00010000:0.0:1788845043.784031:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a009685f80 00000100:00100000:2.0:1788845043.787551:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.787795:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641777664 00000100:00100000:0.0:1788845043.787805:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641777664 00000100:00100000:0.0:1788845043.787858:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1531 00000100:00100000:0.0:1788845043.787867:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641777664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.787917:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.787935:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aebfb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0cf expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.787969:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.788009:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660387072:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.788080:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18480 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.790321:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660387072:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.790344:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.790361:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aebfb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0cf expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.790381:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.790383:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.790388:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aebfb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0cf expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.790401:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006600, rc 0) 00000100:00100000:2.0:1788845043.790487:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641777664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2621us (2954us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.790496:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1531 00000100:00100000:3.0:1788845043.792933:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.793084:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641777792 00000100:00100000:0.0:1788845043.793094:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641777792 00000100:00100000:0.0:1788845043.793163:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1532 00000100:00100000:0.0:1788845043.793174:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641777792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.793206:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.793227:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aec02 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0d6 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.793266:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.793305:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660387200:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.793386:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.796509:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660387200:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845043.796547:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:0.0:1788845043.796570:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aec02 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0d6 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.796591:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.796594:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845043.796598:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aec02 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0d6 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.796614:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845043.796690:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641777792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3516us (3763us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.796698:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1532 00000100:00100000:2.0:1788845043.799804:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.799917:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641777920 00000100:00100000:1.0:1788845043.799926:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641777920 00000100:00100000:1.0:1788845043.799983:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1113 00000100:00100000:1.0:1788845043.799994:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641777920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845043.800013:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845043.800032:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287aec09 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0dd expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.800090:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660387328:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845043.800167:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.802752:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660387328:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845043.802793:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287aec09 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0dd expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.802817:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055bf923b/0xb2187301287aec09 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0dd expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.802829:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f01d14800, rc 0) 00000100:00100000:1.0:1788845043.802916:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2367:x1875739641777920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2923us (3121us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.802925:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1113 00000100:00100000:3.0:1788845043.806853:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.806991:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641778048 00000100:00100000:2.0:1788845043.807002:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641778048 00000100:00100000:2.0:1788845043.807057:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1114 00000100:00100000:2.0:1788845043.807067:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641778048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.807086:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.807117:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0e4 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.807150:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.807158:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(2 17) 00010000:00010000:2.0:1788845043.807162:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:2.0:1788845043.807168:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1683456->1687551] (req 1683456->1687551) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.807221:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 5 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.807240:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 5 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.807266:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660387456:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845043.807346:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.811160:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.811374:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641778176 00000100:00100000:1.0:1788845043.811385:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641778176 00002000:00100000:1.0:1788845043.811441:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1277951) req@ffff94a035328000 x1875739641778176/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.811453:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035328000 with x1875739641778176 ext(1216512->1277951) 00010000:00010000:1.0:1788845043.811463:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 6 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.811484:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 6 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845043.811493:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a035328000 00000100:00100000:1.0:1788845043.811527:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 32 00000100:00100000:1.0:1788845043.811547:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641778176:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845043.815688:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660387456:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845043.815732:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1683456->1687551] (req 1683456->1687551) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.815755:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:3.0:1788845043.816770:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641778176:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5218us (5613us total) trans 4294967494 rc 0/0 00000100:00100000:3.0:1788845043.816782:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 32 00002000:00100000:3.0:1788845043.816795:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1277951) req@ffff94a035328000 x1875739641778176/t4294967494(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/448 e 0 to 0 dl 1788845054 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.816807:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035328000 with x1875739641778176 ext(1216512->1277951) 00010000:00010000:3.0:1788845043.816817:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 8 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.816838:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 8 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845043.816846:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a035328000 00000100:00100000:0.0:1788845043.820754:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.820874:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641778304 00000100:00100000:1.0:1788845043.820908:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641778304 00010000:00010000:1.0:1788845043.820952:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 8 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.820990:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 516 00000100:00100000:1.0:1788845043.821000:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d12450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641778304:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845043.821016:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845043.821038:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 8 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.821057:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 8 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.821083:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005effdd11/0xb2187301287aebae lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1216512->1277951) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa089 expref: 8 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.821102:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:1.0:1788845043.821107:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:1.0:1788845043.821116:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.821127:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.821166:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845043.821187:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660387584:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845043.821261:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.822963:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641778048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15888us (16109us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845043.822975:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1114 00000100:00100000:0.0:1788845043.827481:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.827594:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660387584:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845043.827619:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845043.827689:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d12450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641778304:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6690us (6945us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845043.827700:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 516 00000100:00100000:3.0:1788845043.829731:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641778432 00000100:00100000:3.0:1788845043.829746:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641778432 00002000:00100000:3.0:1788845043.829805:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1683456->2084863) req@ffff949f01d8c000 x1875739641778432/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 504/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.829817:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8c000 with x1875739641778432 ext(1683456->2084863) 00002000:00010000:3.0:1788845043.829823:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8c000 00000100:00100000:3.0:1788845043.829863:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 531 00000100:00100000:3.0:1788845043.829873:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641778432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.833594:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641778432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 3711us (6111us total) trans 0 rc 36864/36864 00000100:00100000:3.0:1788845043.833608:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 531 00002000:00100000:3.0:1788845043.833621:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1683456->2084863) req@ffff949f01d8c000 x1875739641778432/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 504/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.833632:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8c000 with x1875739641778432 ext(1683456->2084863) 00002000:00010000:3.0:1788845043.833638:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8c000 00000100:00100000:0.0:1788845043.837030:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.837229:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641778560 00000100:00100000:2.0:1788845043.837239:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641778560 00002000:00100000:2.0:1788845043.837291:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1687552->1691647) req@ffff949f01d8ea00 x1875739641778560/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.837302:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8ea00 with x1875739641778560 ext(1687552->1691647) 00002000:00010000:2.0:1788845043.837308:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8ea00 00000100:00100000:2.0:1788845043.837342:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 532 00000100:00100000:2.0:1788845043.837352:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641778560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845043.837601:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641778560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (580us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845043.837609:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 532 00002000:00100000:2.0:1788845043.837616:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1687552->1691647) req@ffff949f01d8ea00 x1875739641778560/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.837624:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8ea00 with x1875739641778560 ext(1687552->1691647) 00002000:00010000:2.0:1788845043.837627:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8ea00 00000100:00100000:2.0:1788845043.842014:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.842113:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641778688 00000100:00100000:3.0:1788845043.842124:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641778688 00002000:00100000:3.0:1788845043.842176:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1691648->1695743) req@ffff94a03532aa00 x1875739641778688/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.842199:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03532aa00 with x1875739641778688 ext(1691648->1695743) 00002000:00010000:3.0:1788845043.842205:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03532aa00 00000100:00100000:3.0:1788845043.842230:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 533 00000100:00100000:3.0:1788845043.842239:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641778688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.842494:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641778688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (491us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.842502:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 533 00002000:00100000:3.0:1788845043.842510:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1691648->1695743) req@ffff94a03532aa00 x1875739641778688/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.842517:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03532aa00 with x1875739641778688 ext(1691648->1695743) 00002000:00010000:3.0:1788845043.842520:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03532aa00 00000100:00100000:0.0:1788845043.847860:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.847939:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.848152:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641778944 00000100:00100000:2.0:1788845043.848161:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641778944 00000100:00100000:2.0:1788845043.848220:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1115 00000100:00100000:2.0:1788845043.848231:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641778944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.848250:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.848267:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0f2 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.848291:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:2.0:1788845043.848296:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:2.0:1788845043.848306:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.848322:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.848335:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:2.0:1788845043.848394:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641778944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 164us (542us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845043.848401:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1115 00000100:00100000:3.0:1788845043.849917:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641778816 00000100:00100000:3.0:1788845043.849932:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641778816 00000100:00100000:0.0:1788845043.851099:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1533 00000100:00100000:0.0:1788845043.851124:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641778816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.851145:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.851167:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287aec1e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0eb expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.851332:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.851374:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8d500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660387712:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.851462:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.853790:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8d500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660387712:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845043.853814:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:1.0:1788845043.853832:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287aec1e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0eb expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.853851:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845043.853854:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845043.853859:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287aec1e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0eb expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845043.853873:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:1.0:1788845043.853976:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641778816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2855us (6034us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845043.853984:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1533 00000100:00100000:3.0:1788845043.856616:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.857402:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641779072 00000100:00100000:2.0:1788845043.857413:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641779072 00000100:00100000:2.0:1788845043.857477:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1534 00000100:00100000:2.0:1788845043.857488:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641779072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.857509:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.857682:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aec25 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa0f9 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.857727:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.857770:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660387840:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.857855:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.859989:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660387840:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845043.860016:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:2.0:1788845043.860034:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aec25 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f9 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.860055:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.860058:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.860063:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aec25 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f9 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.860080:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e800, rc 0) 00000100:00100000:2.0:1788845043.860429:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641779072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2941us (3820us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.860438:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1534 00000100:00100000:2.0:1788845043.865425:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.865524:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641779200 00000100:00100000:3.0:1788845043.865542:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641779200 00002000:00100000:3.0:1788845043.865592:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1806336->1810431) req@ffff94a035329180 x1875739641779200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.865604:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329180 with x1875739641779200 ext(1806336->1810431) 00002000:00010000:3.0:1788845043.865609:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329180 00000100:00100000:3.0:1788845043.865687:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 534 00000100:00100000:3.0:1788845043.865698:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641779200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.865966:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641779200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (550us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.865974:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 534 00002000:00100000:3.0:1788845043.865981:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1806336->1810431) req@ffff94a035329180 x1875739641779200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.865989:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329180 with x1875739641779200 ext(1806336->1810431) 00002000:00010000:3.0:1788845043.865993:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329180 00000100:00100000:2.0:1788845043.869946:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.870045:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641779328 00000100:00100000:3.0:1788845043.870055:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641779328 00002000:00100000:3.0:1788845043.870528:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1810432->1814527) req@ffff94a035329880 x1875739641779328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.870601:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329880 with x1875739641779328 ext(1810432->1814527) 00002000:00010000:3.0:1788845043.870608:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329880 00000100:00100000:3.0:1788845043.870705:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 535 00000100:00100000:3.0:1788845043.870717:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641779328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.871035:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641779328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 319us (1099us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845043.871044:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 535 00002000:00100000:3.0:1788845043.871052:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1810432->1814527) req@ffff94a035329880 x1875739641779328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845043.871060:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329880 with x1875739641779328 ext(1810432->1814527) 00002000:00010000:3.0:1788845043.871063:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329880 00000100:00100000:0.0:1788845043.874370:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.874472:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641779456 00000100:00100000:1.0:1788845043.874482:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641779456 00002000:00100000:1.0:1788845043.874531:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1818623) req@ffff949f01d8f480 x1875739641779456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.874614:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8f480 with x1875739641779456 ext(1814528->1818623) 00002000:00010000:1.0:1788845043.874620:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8f480 00000100:00100000:1.0:1788845043.874696:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 536 00000100:00100000:1.0:1788845043.874708:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641779456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845043.874966:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641779456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (604us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845043.874975:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 536 00002000:00100000:1.0:1788845043.874982:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1818623) req@ffff949f01d8f480 x1875739641779456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845043.874990:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8f480 with x1875739641779456 ext(1814528->1818623) 00002000:00010000:1.0:1788845043.874994:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8f480 00000100:00100000:2.0:1788845043.877498:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.877597:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641779584 00000100:00100000:0.0:1788845043.877607:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641779584 00002000:00100000:0.0:1788845043.877654:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1818624->1822719) req@ffff94a03532a300 x1875739641779584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.877677:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03532a300 with x1875739641779584 ext(1818624->1822719) 00002000:00010000:0.0:1788845043.877682:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03532a300 00000100:00100000:0.0:1788845043.877716:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 537 00000100:00100000:0.0:1788845043.877727:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641779584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845043.877972:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641779584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 247us (483us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845043.877981:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 537 00002000:00100000:0.0:1788845043.877987:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1818624->1822719) req@ffff94a03532a300 x1875739641779584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.878003:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03532a300 with x1875739641779584 ext(1818624->1822719) 00002000:00010000:0.0:1788845043.878006:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03532a300 00000100:00100000:2.0:1788845043.885791:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845043.885978:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641779712 00000100:00100000:1.0:1788845043.885987:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641779712 00000100:00100000:1.0:1788845043.886063:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 995 00000100:00100000:1.0:1788845043.886075:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641779712:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:1.0:1788845043.886627:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 912867 -> 912867 00000004:00010000:1.0:1788845043.886633:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00000004:00010000:1.0:1788845043.886635:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:1.0:1788845043.886638:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00000004:00010000:1.0:1788845043.886640:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:1.0:1788845043.886765:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641779712:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 691us (981us total) trans 4294967841 rc 0/0 00000100:00100000:1.0:1788845043.886773:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 995 00000100:00100000:2.0:1788845043.890258:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.890798:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641779840 00000100:00100000:0.0:1788845043.890808:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641779840 00000100:00100000:0.0:1788845043.891009:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1535 00000100:00100000:0.0:1788845043.891021:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641779840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.891045:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.891069:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287aec2c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa100 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.891116:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.891157:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660387968:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.891370:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.893405:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660387968:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845043.893430:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:0.0:1788845043.893449:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287aec2c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa100 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.893469:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.893472:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845043.893478:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287aec2c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa100 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.893494:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845043.893617:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641779840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2597us (3368us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.893625:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1535 00000100:00100000:1.0:1788845043.897231:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.897376:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641779968 00000100:00100000:0.0:1788845043.897387:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641779968 00000100:00100000:0.0:1788845043.897488:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1536 00000100:00100000:0.0:1788845043.897501:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641779968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.897522:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.897607:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aec33 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa107 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.897647:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.897698:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660388096:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845043.897784:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.899925:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660388096:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845043.899951:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:0.0:1788845043.899970:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aec33 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa107 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.899991:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.899994:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845043.899998:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287aec33 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa107 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.900015:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845043.900072:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641779968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2573us (2975us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845043.900080:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1536 00000100:00100000:3.0:1788845043.904022:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.904119:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641780096 00000100:00100000:2.0:1788845043.904131:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641780096 00000100:00100000:2.0:1788845043.904202:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1537 00000100:00100000:2.0:1788845043.904212:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641780096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.904234:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.904256:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aec3a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa10e expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.904289:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.904300:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aec3a lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa10e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.904333:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.904351:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 17 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.904365:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660388224:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845043.904502:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.909239:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.909428:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641780224 00000100:00100000:0.0:1788845043.909439:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641780224 00010000:00010000:0.0:1788845043.909479:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 18 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.909520:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 517 00000100:00100000:0.0:1788845043.909539:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641780224:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845043.909555:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845043.909582:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 18 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.909600:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 18 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.909624:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845043.909630:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aec3a lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa10e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.909640:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aec3a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa10e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.909649:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aec3a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa10e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.909697:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aec3a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa10e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.909713:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660388352:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845043.909796:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.912391:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660388352:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845043.912416:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845043.912478:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2370:x1875739641780224:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2949us (3248us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.912488:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 517 00010000:00010000:2.0:1788845043.913596:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031e00/0xb2187301287aeac0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a9fbe expref: 17 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.913623:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660388224:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845043.913665:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aec3a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa10e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.913685:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845043.913688:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845043.913692:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:2.0:1788845043.913757:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641780096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9546us (9745us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845043.913767:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1537 00000100:00100000:1.0:1788845043.924590:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.924713:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641780352 00000100:00100000:2.0:1788845043.924723:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641780352 00002000:00100000:2.0:1788845043.924773:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c2a00 x1875739641780352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.924784:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c2a00 with x1875739641780352 00000100:00100000:2.0:1788845043.924820:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 996 00000100:00100000:2.0:1788845043.924831:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641780352:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845043.931021:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.932233:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641780480 00000100:00100000:2.0:1788845043.932245:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641780480 00002000:00100000:2.0:1788845043.932297:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2084863) req@ffff94a010481f80 x1875739641780480/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 504/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.932308:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010481f80 with x1875739641780480 ext(1048576->2084863) 00002000:00010000:2.0:1788845043.932313:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010481f80 00000100:00100000:2.0:1788845043.932402:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 538 00000100:00100000:2.0:1788845043.932413:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641780480:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845043.946969:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641780352:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 22129us (22381us total) trans 0 rc 913408/913408 00000100:00100000:2.0:1788845043.946985:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 996 00002000:00100000:2.0:1788845043.947000:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c2a00 x1875739641780352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 488/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 913408/913408 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.947011:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c2a00 with x1875739641780352 00000100:00100000:2.0:1788845043.949970:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641780480:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 17548us (18948us total) trans 0 rc 1019904/1019904 00000100:00100000:2.0:1788845043.949984:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 538 00002000:00100000:2.0:1788845043.949997:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2084863) req@ffff94a010481f80 x1875739641780480/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:149/0 lens 504/440 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 1019904/1019904 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845043.950007:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010481f80 with x1875739641780480 ext(1048576->2084863) 00002000:00010000:2.0:1788845043.950013:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010481f80 00000100:00100000:3.0:1788845043.957053:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.957168:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641780608 00000100:00100000:0.0:1788845043.957186:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641780608 00000100:00100000:0.0:1788845043.957239:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1116 00000100:00100000:0.0:1788845043.957249:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641780608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845043.957267:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845043.957286:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa115 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.957318:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.957328:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.957336:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:0.0:1788845043.957340:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:0.0:1788845043.957345:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1871872->1929215] (req 1871872->1929215) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.957379:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.957398:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 7 pid: 7983 timeout: 533 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.957412:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660388480:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845043.957509:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.957519:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 10 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.957529:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660388608:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845043.957563:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845043.960403:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660388608:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845043.961234:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.961521:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641780736 00000100:00100000:0.0:1788845043.961540:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641780736 00010000:00010000:0.0:1788845043.961580:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 11 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845043.961625:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 518 00000100:00100000:0.0:1788845043.961634:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641780736:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845043.961650:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845043.961684:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 11 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.961696:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 11 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.961720:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec10 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1683456->1687551) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0e4 expref: 11 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.961736:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845043.961798:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641780736:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 165us (569us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.961806:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 518 00000100:00100000:0.0:1788845043.964591:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660388480:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845043.964632:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1871872->1929215] (req 1871872->1929215) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845043.964652:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845043.964719:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641780608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7471us (7676us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845043.964728:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1116 00000100:00100000:2.0:1788845043.969022:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845043.969107:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641780864 00000100:00100000:3.0:1788845043.969117:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641780864 00010000:00010000:3.0:1788845043.969157:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 8 pid: 7983 timeout: 533 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.969271:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 519 00000100:00100000:3.0:1788845043.969282:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641780864:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845043.969297:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845043.969321:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 8 pid: 7983 timeout: 533 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.969340:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 8 pid: 7983 timeout: 533 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.969363:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec17 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa0f2 expref: 8 pid: 7983 timeout: 533 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.969380:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:3.0:1788845043.969385:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:3.0:1788845043.969393:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.969410:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845043.969448:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845043.969463:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010481880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660388736:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845043.969688:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845043.977403:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.977483:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641780992 00000100:00100000:2.0:1788845043.977493:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641780992 00000100:00100000:2.0:1788845043.977569:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:2.0:1788845043.977579:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641780992:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:2.0:1788845043.977861:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2081862 -> 2081862 00002000:00010000:2.0:1788845043.977864:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845043 -> 1788845043 00002000:00010000:2.0:1788845043.977867:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00002000:00010000:2.0:1788845043.977868:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845043 -> 1788845043 00002000:00010000:2.0:1788845043.977871:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 57 -> 57 00000100:00100000:2.0:1788845043.977959:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641780992:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 381us (565us total) trans 4294967495 rc 0/0 00000100:00100000:2.0:1788845043.977967:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:3.0:1788845043.980095:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010481880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660388736:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845043.980124:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845043.980186:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641780864:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10906us (11173us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845043.980196:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 519 00000100:00100000:1.0:1788845043.981737:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.981813:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641781120 00000100:00100000:2.0:1788845043.981821:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641781120 00000100:00100000:2.0:1788845043.981948:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1117 00000100:00100000:2.0:1788845043.981958:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641781120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.981977:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.981997:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007ff92ced/0xb2187301287aec48 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa11c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.982053:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660388864:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.982129:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845043.984387:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845043 -> 1788845044 00000100:00100000:2.0:1788845043.984419:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660388864:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.984454:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007ff92ced/0xb2187301287aec48 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa11c expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.984478:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ff92ced/0xb2187301287aec48 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa11c expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.984492:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e800, rc 0) 00000100:00100000:2.0:1788845043.984560:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641781120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2603us (2829us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.984568:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1117 00000100:00100000:0.0:1788845043.987734:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.987813:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641781248 00000100:00100000:2.0:1788845043.987824:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641781248 00000100:00100000:2.0:1788845043.987914:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1118 00000100:00100000:2.0:1788845043.987926:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641781248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845043.987944:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845043.987962:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec4f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa123 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845043.988024:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660388992:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845043.988107:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845043.991588:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035329f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660388992:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845043.991633:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec4f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa123 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.991667:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aec4f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa123 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845043.991680:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:2.0:1788845043.991742:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641781248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3818us (4017us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845043.991751:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1118 00000100:00100000:3.0:1788845043.995981:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845043.996095:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641781376 00000100:00100000:0.0:1788845043.996104:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641781376 00002000:00100000:0.0:1788845043.996153:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1572864->1576959) req@ffff94a010481880 x1875739641781376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.996165:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010481880 with x1875739641781376 ext(1572864->1576959) 00002000:00010000:0.0:1788845043.996171:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010481880 00000100:00100000:0.0:1788845043.996214:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 540 00000100:00100000:0.0:1788845043.996223:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641781376:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845043.996457:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641781376:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 233us (483us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845043.996464:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 540 00002000:00100000:0.0:1788845043.996471:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1572864->1576959) req@ffff94a010481880 x1875739641781376/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845043.996478:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010481880 with x1875739641781376 ext(1572864->1576959) 00002000:00010000:0.0:1788845043.996481:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010481880 00000100:00100000:0.0:1788845043.999283:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845043.999974:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641781504 00000100:00100000:2.0:1788845043.999985:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641781504 00002000:00100000:2.0:1788845044.000035:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1576960->1581055) req@ffff949f01d8df80 x1875739641781504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/0 e 0 to 0 dl 1788845054 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.000053:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8df80 with x1875739641781504 ext(1576960->1581055) 00002000:00010000:2.0:1788845044.000059:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8df80 00000100:00100000:2.0:1788845044.000144:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 541 00000100:00100000:2.0:1788845044.000155:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641781504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845044.000412:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641781504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (1138us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845044.000420:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 541 00002000:00100000:2.0:1788845044.000427:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1576960->1581055) req@ffff949f01d8df80 x1875739641781504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:149/0 lens 488/4536 e 0 to 0 dl 1788845054 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.000436:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8df80 with x1875739641781504 ext(1576960->1581055) 00002000:00010000:2.0:1788845044.000440:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8df80 00000100:00100000:3.0:1788845044.003245:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.003829:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641781632 00000100:00100000:1.0:1788845044.003840:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641781632 00002000:00100000:1.0:1788845044.003902:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1581056->1585151) req@ffff94a010481500 x1875739641781632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845044.003913:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010481500 with x1875739641781632 ext(1581056->1585151) 00002000:00010000:1.0:1788845044.003918:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010481500 00000100:00100000:1.0:1788845044.003987:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 542 00000100:00100000:1.0:1788845044.003997:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641781632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845044.004405:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641781632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 409us (1167us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845044.004413:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 542 00002000:00100000:1.0:1788845044.004420:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1581056->1585151) req@ffff94a010481500 x1875739641781632/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845044.004428:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010481500 with x1875739641781632 ext(1581056->1585151) 00002000:00010000:1.0:1788845044.004432:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010481500 00000100:00100000:3.0:1788845044.008034:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.008148:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641781760 00000100:00100000:2.0:1788845044.008159:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641781760 00002000:00100000:2.0:1788845044.008304:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1585152->1589247) req@ffff94a03f6c5c00 x1875739641781760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.008315:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c5c00 with x1875739641781760 ext(1585152->1589247) 00002000:00010000:2.0:1788845044.008320:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c5c00 00000100:00100000:2.0:1788845044.008402:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 543 00000100:00100000:2.0:1788845044.008414:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641781760:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845044.008646:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641781760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 234us (620us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845044.008662:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 543 00002000:00100000:2.0:1788845044.008669:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1585152->1589247) req@ffff94a03f6c5c00 x1875739641781760/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.008677:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c5c00 with x1875739641781760 ext(1585152->1589247) 00002000:00010000:2.0:1788845044.008680:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c5c00 00000100:00100000:0.0:1788845044.011244:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.011695:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641781888 00000100:00100000:1.0:1788845044.011705:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641781888 00002000:00100000:0.0:1788845044.011819:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1589248->1593343) req@ffff949f01d8d500 x1875739641781888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845044.011830:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8d500 with x1875739641781888 ext(1589248->1593343) 00002000:00010000:0.0:1788845044.011836:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8d500 00000100:00100000:0.0:1788845044.011919:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 544 00000100:00100000:0.0:1788845044.011928:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641781888:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845044.012152:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2367:x1875739641781888:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 224us (916us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845044.012164:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 544 00002000:00100000:0.0:1788845044.012170:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1589248->1593343) req@ffff949f01d8d500 x1875739641781888/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845044.012296:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8d500 with x1875739641781888 ext(1589248->1593343) 00002000:00010000:0.0:1788845044.012300:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8d500 00000100:00100000:2.0:1788845044.015619:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.015707:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641782016 00000100:00100000:0.0:1788845044.015717:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641782016 00002000:00100000:0.0:1788845044.015769:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1597439) req@ffff94a035329f80 x1875739641782016/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845044.015779:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329f80 with x1875739641782016 ext(1593344->1597439) 00002000:00010000:0.0:1788845044.015785:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329f80 00000100:00100000:0.0:1788845044.015819:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 545 00000100:00100000:0.0:1788845044.015830:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641782016:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845044.016385:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641782016:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 556us (774us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845044.016394:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 545 00002000:00100000:0.0:1788845044.016401:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1597439) req@ffff94a035329f80 x1875739641782016/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845044.016409:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329f80 with x1875739641782016 ext(1593344->1597439) 00002000:00010000:0.0:1788845044.016412:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329f80 00000100:00100000:2.0:1788845044.019944:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.020032:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641782144 00000100:00100000:3.0:1788845044.020043:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641782144 00002000:00100000:3.0:1788845044.020100:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1601535) req@ffff94a035328380 x1875739641782144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.020113:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035328380 with x1875739641782144 ext(1597440->1601535) 00002000:00010000:3.0:1788845044.020118:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035328380 00000100:00100000:3.0:1788845044.020151:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 546 00000100:00100000:3.0:1788845044.020161:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641782144:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845044.020406:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641782144:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 246us (470us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845044.020414:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 546 00002000:00100000:3.0:1788845044.020420:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1597440->1601535) req@ffff94a035328380 x1875739641782144/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.020429:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035328380 with x1875739641782144 ext(1597440->1601535) 00002000:00010000:3.0:1788845044.020432:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035328380 00000100:00100000:0.0:1788845044.023863:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.023984:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641782272 00000100:00100000:3.0:1788845044.023993:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641782272 00002000:00100000:3.0:1788845044.024043:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1601536->1605631) req@ffff949f01d8d880 x1875739641782272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.024053:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8d880 with x1875739641782272 ext(1601536->1605631) 00002000:00010000:3.0:1788845044.024059:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8d880 00000100:00100000:3.0:1788845044.024090:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 547 00000100:00100000:3.0:1788845044.024101:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641782272:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845044.024347:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641782272:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 247us (492us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845044.024354:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 547 00002000:00100000:3.0:1788845044.024361:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1601536->1605631) req@ffff949f01d8d880 x1875739641782272/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.024369:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8d880 with x1875739641782272 ext(1601536->1605631) 00002000:00010000:3.0:1788845044.024372:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8d880 00000100:00100000:0.0:1788845044.027848:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.027938:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641782400 00000100:00100000:3.0:1788845044.027950:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641782400 00002000:00100000:3.0:1788845044.028004:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1605632->1609727) req@ffff949f01d8c700 x1875739641782400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.028015:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8c700 with x1875739641782400 ext(1605632->1609727) 00002000:00010000:3.0:1788845044.028022:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8c700 00000100:00100000:3.0:1788845044.028057:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 548 00000100:00100000:3.0:1788845044.028067:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641782400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845044.028309:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641782400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (469us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845044.028318:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 548 00002000:00100000:3.0:1788845044.028325:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1605632->1609727) req@ffff949f01d8c700 x1875739641782400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.028334:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01d8c700 with x1875739641782400 ext(1605632->1609727) 00002000:00010000:3.0:1788845044.028337:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01d8c700 00000100:00100000:2.0:1788845044.033664:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.033789:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641782528 00000100:00100000:1.0:1788845044.033799:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641782528 00000100:00100000:1.0:1788845044.033935:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1538 00000100:00100000:1.0:1788845044.033946:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641782528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.033972:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.033993:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa12a expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.034030:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845044.034043:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 912867 00010000:00010000:1.0:1788845044.034050:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.034066:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845044.034068:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845044.034072:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756e00, rc 0) 00000100:00100000:1.0:1788845044.034127:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641782528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 184us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845044.034134:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1538 00000100:00100000:0.0:1788845044.037636:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.037742:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641782656 00000100:00100000:1.0:1788845044.037752:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641782656 00000100:00100000:1.0:1788845044.037815:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1119 00000100:00100000:1.0:1788845044.037825:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641782656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.037847:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.037867:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa131 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.037924:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.037939:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 17) 00010000:00010000:1.0:1788845044.037943:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 17) 00010000:00010000:1.0:1788845044.037947:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1273856->1290239] (req 1273856->1290239) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.037983:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 9 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.038001:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 9 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.038015:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660389120:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845044.038567:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.043450:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.043541:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641782784 00000100:00100000:3.0:1788845044.043552:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641782784 00010000:00010000:3.0:1788845044.043592:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 10 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.043639:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 520 00000100:00100000:3.0:1788845044.043650:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641782784:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845044.043675:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.043700:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 10 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.043724:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 10 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.043750:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 17) 00010000:00010000:3.0:1788845044.043755:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 17) 00010000:00010000:3.0:1788845044.043763:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.043774:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.043812:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.043826:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660389248:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845044.043927:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.046388:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660389248:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845044.046415:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845044.046472:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641782784:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2823us (3034us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.046482:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 520 00010000:00010000:1.0:1788845044.047812:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287aec41 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1871872->1929215) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa115 expref: 9 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.047834:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660389120:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845044.047860:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 8 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.047875:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756600, rc 0) 00000100:00100000:0.0:1788845044.048086:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:14891:x1875739641782656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10261us (10458us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.048094:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1119 00000100:00100000:2.0:1788845044.051397:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.051635:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641782912 00000100:00100000:3.0:1788845044.051646:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641782912 00002000:00100000:3.0:1788845044.051709:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a035329c00 x1875739641782912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.051722:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329c00 with x1875739641782912 ext(1273856->1277951) 00002000:00010000:3.0:1788845044.051728:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329c00 00000100:00100000:3.0:1788845044.051764:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 549 00000100:00100000:3.0:1788845044.051775:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035329c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641782912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845044.052117:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035329c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641782912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 343us (728us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845044.052125:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 549 00002000:00100000:3.0:1788845044.052132:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff94a035329c00 x1875739641782912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.052140:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035329c00 with x1875739641782912 ext(1273856->1277951) 00002000:00010000:3.0:1788845044.052143:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035329c00 00000100:00100000:1.0:1788845044.056062:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.056161:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641783040 00000100:00100000:3.0:1788845044.056172:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641783040 00002000:00100000:3.0:1788845044.056232:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1286144->1290239) req@ffff94a0360ef480 x1875739641783040/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.056244:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ef480 with x1875739641783040 ext(1286144->1290239) 00002000:00010000:3.0:1788845044.056250:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ef480 00000100:00100000:3.0:1788845044.056279:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 550 00000100:00100000:3.0:1788845044.056290:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641783040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845044.056541:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641783040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 252us (488us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845044.056549:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 550 00002000:00100000:3.0:1788845044.056558:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1286144->1290239) req@ffff94a0360ef480 x1875739641783040/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.056566:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ef480 with x1875739641783040 ext(1286144->1290239) 00002000:00010000:3.0:1788845044.056569:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ef480 00000100:00100000:0.0:1788845044.060839:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.060941:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641783168 00000100:00100000:3.0:1788845044.060950:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641783168 00000100:00100000:3.0:1788845044.061010:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1120 00000100:00100000:3.0:1788845044.061021:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641783168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.061040:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.061060:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287aec64 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa138 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.061117:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660389376:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.061432:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22f00 going to sleep for 11 seconds 00002000:00010000:3.0:1788845044.064634:0:6685:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845043 -> 1788845044 00000100:00100000:3.0:1788845044.064676:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660389376:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.064711:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287aec64 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa138 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.064736:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287aec64 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa138 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.064751:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:3.0:1788845044.064845:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641783168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3825us (4013us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.064857:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1120 00000100:00100000:0.0:1788845044.067750:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.067855:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641783296 00000100:00100000:2.0:1788845044.067865:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641783296 00000100:00100000:2.0:1788845044.068011:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1121 00000100:00100000:2.0:1788845044.068022:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641783296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.068040:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.068058:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec6b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa13f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.068116:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03532b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660389504:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845044.068400:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.075149:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03532b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660389504:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845044.075207:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec6b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa13f expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.075233:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000095aef19/0xb2187301287aec6b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa13f expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.075246:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:1.0:1788845044.075310:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641783296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7290us (7569us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.075319:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1121 00000100:00100000:0.0:1788845044.078938:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.079056:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641783424 00000100:00100000:3.0:1788845044.079067:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641783424 00000100:00100000:3.0:1788845044.079125:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1122 00000100:00100000:3.0:1788845044.079136:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641783424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.079153:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.079183:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287aec72 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa146 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.079240:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660389632:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.079327:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.082195:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660389632:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.082235:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287aec72 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa146 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.082270:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287aec72 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa146 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.082285:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845044.082341:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641783424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3207us (3412us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.082348:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1122 00000100:00100000:1.0:1788845044.085277:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.085424:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641783552 00000100:00100000:0.0:1788845044.085434:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641783552 00000100:00100000:0.0:1788845044.085490:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1123 00000100:00100000:0.0:1788845044.085501:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641783552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.085520:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.085548:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aec79 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa14d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.085607:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660389760:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.085702:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.087778:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660389760:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845044.087817:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aec79 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa14d expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.087842:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aec79 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa14d expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.087859:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:2.0:1788845044.087952:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641783552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2452us (2694us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.087961:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1123 00000100:00100000:2.0:1788845044.094075:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.094162:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641783680 00000100:00100000:0.0:1788845044.094182:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641783680 00000100:00100000:0.0:1788845044.094242:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1539 00000100:00100000:0.0:1788845044.094252:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641783680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.094473:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.094477:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788845044.094522:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aec3a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa10e expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.094553:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.094560:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845044.094574:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa154 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.094600:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.094612:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.094649:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.094682:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 17 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.094697:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660389888:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845044.094834:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18600 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.099715:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.099810:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641783808 00000100:00100000:2.0:1788845044.099820:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641783808 00010000:00010000:2.0:1788845044.099860:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 18 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.099969:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 521 00000100:00100000:2.0:1788845044.099981:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641783808:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845044.099996:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845044.100023:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 18 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.100041:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 18 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.100062:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.100068:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.100081:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.100095:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.100142:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.100157:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02362a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660390016:192.168.201.29@tcp:105: 00000100:00100000:2.0:1788845044.100243:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00010000:00010000:0.0:1788845044.101100:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756e00/0xb2187301287aec56 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa12a expref: 18 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.101120:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660389888:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845044.101149:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.101162:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.101165:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.101169:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845044.101227:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641783680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6977us (7163us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.101236:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1539 00000100:00100000:2.0:1788845044.104105:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02362a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660390016:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845044.104129:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845044.104196:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641783808:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4217us (4490us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845044.104206:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 521 00000100:00100000:2.0:1788845044.107432:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.107525:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641783936 00000100:00100000:0.0:1788845044.107591:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641783936 00002000:00100000:0.0:1788845044.107648:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02362a00 x1875739641783936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845044.107669:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02362a00 with x1875739641783936 00000100:00100000:0.0:1788845044.107765:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 997 00000100:00100000:0.0:1788845044.107775:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641783936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845044.108063:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641783936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 288us (639us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845044.108073:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 997 00002000:00100000:0.0:1788845044.108080:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02362a00 x1875739641783936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845044.108088:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02362a00 with x1875739641783936 00000100:00100000:1.0:1788845044.112257:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.112409:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641784064 00000100:00100000:3.0:1788845044.112420:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641784064 00002000:00100000:3.0:1788845044.112475:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384b800 x1875739641784064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.112487:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384b800 with x1875739641784064 00000100:00100000:3.0:1788845044.112524:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 998 00000100:00100000:3.0:1788845044.112544:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641784064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845044.112795:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641784064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 252us (548us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.112803:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 998 00002000:00100000:3.0:1788845044.112810:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384b800 x1875739641784064/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/440 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.112819:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384b800 with x1875739641784064 00000100:00100000:2.0:1788845044.116674:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.116718:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.116796:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641784192 00000100:00100000:3.0:1788845044.116805:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641784192 00000100:00100000:3.0:1788845044.116865:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1540 00000100:00100000:3.0:1788845044.116874:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641784192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.116925:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.116946:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287aec87 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa15b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.116983:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.117025:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660390144:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.117104:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.117750:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641784320 00000100:00100000:0.0:1788845044.117755:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641784320 00000100:00100000:0.0:1788845044.117798:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1124 00000100:00100000:0.0:1788845044.117806:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641784320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.117818:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.117829:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aec8e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa162 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.117865:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660390272:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.117945:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000004:00010000:3.0:1788845044.120763:0:5841:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 912867 -> 962463 00000004:00010000:3.0:1788845044.120772:0:5841:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845043 -> 1788845044 00000004:00010000:3.0:1788845044.120775:0:5841:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845043 -> 1788845044 00000100:00100000:3.0:1788845044.120800:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660390144:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845044.120824:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:3.0:1788845044.120844:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287aec87 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa15b expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.120865:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.120869:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.120875:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031600/0xb2187301287aec87 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa15b expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.120926:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:3.0:1788845044.120986:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641784192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4112us (4266us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.120995:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1540 00000100:00100000:0.0:1788845044.123847:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660390272:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845044.123937:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aec8e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa162 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.123965:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287aec8e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa162 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.123979:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845044.124042:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641784320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6235us (7374us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.124051:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1124 00000100:00100000:2.0:1788845044.129029:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.129074:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.129147:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641784448 00000100:00100000:0.0:1788845044.129157:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641784448 00000100:00100000:0.0:1788845044.129233:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1541 00000100:00100000:0.0:1788845044.129242:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641784448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.129272:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.129291:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aec95 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa169 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.129329:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.129372:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660390400:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.129452:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.129568:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641784576 00000100:00100000:1.0:1788845044.129573:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641784576 00000100:00100000:1.0:1788845044.129611:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1125 00000100:00100000:1.0:1788845044.129619:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641784576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.129627:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.129637:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000b0717d6/0xb2187301287aec9c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa170 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.129687:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660390528:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.129714:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.132554:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660390400:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845044.132581:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:0.0:1788845044.132600:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aec95 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa169 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.132623:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.132626:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.132631:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aec95 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa169 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.132649:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006600, rc 0) 00000100:00100000:0.0:1788845044.132719:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641784448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3477us (3698us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.132727:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1541 00000100:00100000:1.0:1788845044.133141:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660390528:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845044.133163:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000b0717d6/0xb2187301287aec9c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa170 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.133180:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000b0717d6/0xb2187301287aec9c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa170 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.133191:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757800, rc 0) 00000100:00100000:1.0:1788845044.133223:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641784576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3605us (4149us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.133230:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1125 00000100:00100000:2.0:1788845044.137923:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.137968:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.138119:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641784704 00000100:00100000:1.0:1788845044.138130:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641784704 00000100:00100000:1.0:1788845044.138204:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1542 00000100:00100000:1.0:1788845044.138215:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641784704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.138236:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.138268:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aeca3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa177 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.138306:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.138345:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660390656:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.138427:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.138694:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641784832 00000100:00100000:3.0:1788845044.138699:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641784832 00000100:00100000:3.0:1788845044.138746:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1126 00000100:00100000:3.0:1788845044.138755:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641784832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.138766:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.138779:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287aecaa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa17e expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.138819:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660390784:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.138850:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.142195:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660390784:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845044.142240:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287aecaa lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa17e expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.142284:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ed3d1544/0xb2187301287aecaa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa17e expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.142300:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031600, rc 0) 00000100:00100000:2.0:1788845044.142364:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641784832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3610us (4487us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.142372:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1126 00000100:00100000:1.0:1788845044.142626:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da49f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660390656:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845044.142640:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:1.0:1788845044.142661:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aeca3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa177 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.142681:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845044.142684:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845044.142689:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aeca3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa177 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.142700:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757e00, rc 0) 00000100:00100000:1.0:1788845044.142730:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641784704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4518us (4761us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.142736:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1542 00000100:00100000:2.0:1788845044.148026:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.148169:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.149010:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641785088 00000100:00100000:3.0:1788845044.149021:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641785088 00000100:00100000:3.0:1788845044.149076:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1127 00000100:00100000:3.0:1788845044.149086:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641785088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.149116:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.149136:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287aecb1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa18c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.149209:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660390912:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.149310:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.151624:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660390912:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845044.151674:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287aecb1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa18c expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.151700:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ddc1167/0xb2187301287aecb1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa18c expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.151715:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031e00, rc 0) 00000100:00100000:1.0:1788845044.151779:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2369:x1875739641785088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2693us (3609us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.151788:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1127 00000100:00100000:0.0:1788845044.152674:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641784960 00000100:00100000:0.0:1788845044.152682:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641784960 00000100:00100000:0.0:1788845044.152735:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1543 00000100:00100000:0.0:1788845044.152745:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641784960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.152766:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.152785:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aecb8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa185 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.152821:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.152858:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660391040:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.152950:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.158716:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660391040:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845044.158744:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:0.0:1788845044.158767:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aecb8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa185 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.158788:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.158792:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.158797:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aecb8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa185 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.158812:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006600, rc 0) 00000100:00100000:0.0:1788845044.159003:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641784960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6258us (11134us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.159012:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1543 00000100:00100000:1.0:1788845044.171763:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.171966:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641785216 00000100:00100000:3.0:1788845044.171977:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641785216 00000100:00100000:3.0:1788845044.172039:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1544 00000100:00100000:3.0:1788845044.172052:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641785216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.172072:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.172091:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aecbf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa193 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.172144:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.172183:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c6680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660391168:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.172751:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e229c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.188482:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.188614:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641785344 00000100:00100000:0.0:1788845044.188623:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641785344 00000100:00100000:0.0:1788845044.188691:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1128 00000100:00100000:0.0:1788845044.188701:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034508450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641785344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.188720:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.188738:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009ee626d9/0xb2187301287aecc6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa19a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.188802:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660391296:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.188914:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.189315:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c6680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660391168:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845044.189335:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:3.0:1788845044.189351:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aecbf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa193 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.189370:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.189374:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.189380:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030a00/0xb2187301287aecbf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa193 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.189397:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:3.0:1788845044.189442:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641785216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17395us (17691us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.189450:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1544 00000100:00100000:0.0:1788845044.192610:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660391296:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845044.192649:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009ee626d9/0xb2187301287aecc6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa19a expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.192682:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009ee626d9/0xb2187301287aecc6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa19a expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.192695:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006600, rc 0) 00000100:00100000:0.0:1788845044.192773:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034508450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641785344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4072us (4299us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.192781:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1128 00000100:00100000:2.0:1788845044.200672:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.200727:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.200817:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641785600 00000100:00100000:1.0:1788845044.200827:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641785600 00000100:00100000:1.0:1788845044.200922:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1129 00000100:00100000:1.0:1788845044.200933:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641785600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.200951:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.200971:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b34d57a8/0xb2187301287aeccd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1a8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.201028:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660391424:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.201166:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.201594:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641785472 00000100:00100000:0.0:1788845044.201600:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641785472 00000100:00100000:0.0:1788845044.201643:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1545 00000100:00100000:0.0:1788845044.201661:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641785472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.201674:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.201688:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aecd4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1a1 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.201718:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.201744:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cef100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660391552:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.201787:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.212211:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660391424:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845044.212511:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b34d57a8/0xb2187301287aeccd lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1a8 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.212553:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b34d57a8/0xb2187301287aeccd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1a8 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.212569:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756a00, rc 0) 00000100:00100000:0.0:1788845044.212741:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cef100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660391552:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845044.212757:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:0.0:1788845044.212770:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aecd4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1a1 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.212790:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.212793:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.212798:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287aecd4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1a1 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.212809:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006600, rc 0) 00000100:00100000:1.0:1788845044.212852:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641785600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11921us (12123us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.212861:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1129 00000100:00100000:0.0:1788845044.216794:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641785472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15130us (16131us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.216809:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1545 00000100:00100000:1.0:1788845044.226489:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.226775:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641785728 00000100:00100000:0.0:1788845044.226786:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641785728 00000100:00100000:0.0:1788845044.226849:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1546 00000100:00100000:0.0:1788845044.226860:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641785728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.226912:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.226936:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aecdb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1af expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.226977:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.227019:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cef100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660391680:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.227114:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.229675:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.233642:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641785856 00000100:00100000:3.0:1788845044.233668:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641785856 00000100:00100000:3.0:1788845044.233733:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1130 00000100:00100000:3.0:1788845044.233745:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641785856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.233763:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.233784:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aece2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1b6 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.233843:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660391808:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.234127:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e229c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.239355:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cef100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660391680:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845044.239382:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:0.0:1788845044.239401:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aecdb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1af expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.239425:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.239428:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.239432:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aecdb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1af expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.239449:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845044.239507:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641785728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12649us (13028us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.239516:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1546 00000100:00100000:2.0:1788845044.246001:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660391808:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845044.246051:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aece2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1b6 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.246078:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025d15be6/0xb2187301287aece2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1b6 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.246234:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030a00, rc 0) 00000100:00100000:2.0:1788845044.246358:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641785856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12614us (16690us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.246371:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1130 00000100:00100000:0.0:1788845044.258440:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.258537:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.258678:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641786112 00000100:00100000:3.0:1788845044.258688:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641786112 00000100:00100000:3.0:1788845044.258746:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1131 00000100:00100000:3.0:1788845044.258757:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641786112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.258776:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.258794:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287aece9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1c4 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.258856:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660391936:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.258979:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e229c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.262337:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660391936:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.262377:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287aece9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1c4 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.262403:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287aece9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1c4 expref: 10 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.262417:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:3.0:1788845044.262478:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641786112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3723us (3952us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.262487:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1131 00000100:00100000:2.0:1788845044.266396:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641785984 00000100:00100000:2.0:1788845044.266413:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641785984 00000100:00100000:2.0:1788845044.266494:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1547 00000100:00100000:2.0:1788845044.266507:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641785984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.266541:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.266567:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aecf0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1bd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.266609:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.266667:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660392064:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845044.266755:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.272034:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660392064:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845044.272335:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:2.0:1788845044.272358:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aecf0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1bd expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.272379:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.272382:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845044.272387:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aecf0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1bd expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.272403:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:2.0:1788845044.272501:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641785984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5995us (14068us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.272510:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1547 00000100:00100000:3.0:1788845044.277012:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.277319:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641786240 00000100:00100000:1.0:1788845044.277331:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641786240 00002000:00100000:1.0:1788845044.277383:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c4a80 x1875739641786240/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845044.277393:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c4a80 with x1875739641786240 00000100:00100000:1.0:1788845044.277476:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 999 00000100:00100000:1.0:1788845044.277487:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641786240:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845044.282151:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962463 -> 962463 00000004:00010000:2.0:1788845044.282161:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.282165:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:2.0:1788845044.282178:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.282181:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845044.282268:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641786240:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4774us (5257us total) trans 4294967842 rc 0/0 00000100:00100000:2.0:1788845044.282280:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 999 00002000:00100000:2.0:1788845044.282293:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c4a80 x1875739641786240/t4294967842(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/448 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.282304:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c4a80 with x1875739641786240 00000100:00100000:3.0:1788845044.297005:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.297122:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.297294:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641786368 00000100:00100000:3.0:1788845044.297304:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641786368 00000100:00100000:3.0:1788845044.297378:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1000 00000100:00100000:3.0:1788845044.297389:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641786368:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845044.297596:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962463 -> 962463 00000004:00010000:3.0:1788845044.297600:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:3.0:1788845044.297603:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845044.297605:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:3.0:1788845044.297608:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845044.297677:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2367:x1875739641786368:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 290us (555us total) trans 4294967843 rc 0/0 00000100:00100000:3.0:1788845044.297689:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1000 00000100:00100000:2.0:1788845044.299715:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641786496 00000100:00100000:2.0:1788845044.299730:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641786496 00002000:00100000:2.0:1788845044.299782:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1290239) req@ffff94a03f6c6d80 x1875739641786496/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 16872/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.299795:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c6d80 with x1875739641786496 ext(1273856->1290239) 00002000:00010000:2.0:1788845044.299800:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c6d80 00000100:00100000:2.0:1788845044.299911:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 551 00000100:00100000:2.0:1788845044.299922:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641786496:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845044.303611:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641786496:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3680us (6604us total) trans 4294967496 rc 0/0 00000100:00100000:2.0:1788845044.303626:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 551 00002000:00100000:2.0:1788845044.303640:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1290239) req@ffff94a03f6c6d80 x1875739641786496/t4294967496(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 16872/448 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.303661:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c6d80 with x1875739641786496 ext(1273856->1290239) 00002000:00010000:2.0:1788845044.303667:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f6c6d80 00000100:00100000:1.0:1788845044.306669:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.307245:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641786624 00000100:00100000:2.0:1788845044.307574:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641786624 00010000:00010000:3.0:1788845044.307641:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845044.307685:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:2.0:1788845044.307696:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641786624:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:2.0:1788845044.307955:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2081862 -> 2081862 00002000:00010000:2.0:1788845044.307959:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00002000:00010000:2.0:1788845044.307961:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00002000:00010000:2.0:1788845044.307962:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00002000:00010000:2.0:1788845044.307963:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 57 -> 57 00000100:00100000:2.0:1788845044.308011:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641786624:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 317us (1360us total) trans 4294967497 rc 0/0 00000100:00100000:2.0:1788845044.308019:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:0.0:1788845044.311279:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.311367:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641786752 00000100:00100000:3.0:1788845044.311376:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641786752 00000100:00100000:3.0:1788845044.311451:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 553 00000100:00100000:3.0:1788845044.311461:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641786752:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:3.0:1788845044.311740:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2081862 -> 2081862 00002000:00010000:3.0:1788845044.311744:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00002000:00010000:3.0:1788845044.311747:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00002000:00010000:3.0:1788845044.311749:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00002000:00010000:3.0:1788845044.311751:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 57 -> 57 00000100:00100000:3.0:1788845044.311814:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641786752:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 354us (544us total) trans 4294967498 rc 0/0 00000100:00100000:3.0:1788845044.311823:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 553 00000100:00100000:0.0:1788845044.315072:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.315157:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641786880 00000100:00100000:3.0:1788845044.315167:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641786880 00000100:00100000:3.0:1788845044.315235:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1548 00000100:00100000:3.0:1788845044.315245:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641786880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.315266:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.315286:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287aecf7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1cb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.315323:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.315367:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660392192:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.315458:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e229c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.316513:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.316610:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641787008 00000100:00100000:0.0:1788845044.316617:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641787008 00000100:00100000:0.0:1788845044.316671:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1132 00000100:00100000:0.0:1788845044.316680:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641787008:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.316693:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.316706:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aecfe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1d2 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.316750:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660392320:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.316811:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.318469:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660392320:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.318469:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660392192:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845044.318492:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:3.0:1788845044.318502:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aecfe lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1d2 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.318503:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287aecf7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1cb expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.318521:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.318523:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aecfe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1d2 expref: 10 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.318524:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845044.318528:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031a00/0xb2187301287aecf7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1cb expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.318543:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031a00, rc 0) 00010000:00010000:3.0:1788845044.318544:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:1.0:1788845044.318593:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641786880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3349us (3532us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.318601:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1548 00000100:00100000:3.0:1788845044.318713:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641787008:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2035us (2205us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.318718:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1132 00000100:00100000:1.0:1788845044.321274:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.321318:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.321373:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641787136 00000100:00100000:2.0:1788845044.321383:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641787136 00000100:00100000:2.0:1788845044.321443:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1549 00000100:00100000:2.0:1788845044.321454:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641787136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.321475:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.321496:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aed05 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1d9 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.321598:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641787264 00000100:00100000:3.0:1788845044.321603:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641787264 00000100:00100000:3.0:1788845044.321641:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1133 00000100:00100000:3.0:1788845044.321658:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641787264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.321668:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.321679:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287aed0c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1e0 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.321728:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660392448:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.321806:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22f00 going to sleep for 11 seconds 00010000:00010000:2.0:1788845044.321983:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.322016:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660392576:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845044.322157:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.324663:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660392448:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.324706:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287aed0c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e0 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.324736:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000835b916c/0xb2187301287aed0c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e0 expref: 10 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.324750:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:3.0:1788845044.324809:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641787264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3160us (3488us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.324816:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1133 00000100:00100000:2.0:1788845044.326863:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660392576:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845044.326921:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962463 00010000:00010000:2.0:1788845044.326943:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aed05 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1d9 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.326964:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.326967:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845044.326971:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aed05 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1d9 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.326985:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:2.0:1788845044.327048:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641787136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5594us (5780us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.327056:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1549 00000100:00100000:0.0:1788845044.330621:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.330769:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641787520 00000100:00100000:1.0:1788845044.330780:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641787520 00000100:00100000:1.0:1788845044.330841:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1550 00000100:00100000:1.0:1788845044.330850:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641787520:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845044.330948:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845044.330974:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287aebbc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa090 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.330997:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845044.331001:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845044.331207:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.331221:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.331229:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.331380:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.331399:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 13 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.331413:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660392704:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845044.331488:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.334499:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.334584:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641787648 00000100:00100000:2.0:1788845044.334592:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641787648 00010000:00010000:2.0:1788845044.334627:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 14 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.334671:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 522 00000100:00100000:2.0:1788845044.334681:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641787648:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845044.334697:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845044.334704:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 14 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.334721:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 14 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.334741:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.334746:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.334756:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.334763:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.334782:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845044.334834:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641787648:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 155us (344us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845044.334842:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 522 00010000:00010000:0.0:1788845044.335407:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aeb6f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa04a expref: 13 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.335422:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660392704:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845044.335441:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.335449:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.335455:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845044.335577:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967844 00000004:00010000:0.0:1788845044.335628:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962463 -> 962463 00000004:00010000:0.0:1788845044.335631:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:0.0:1788845044.335633:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:0.0:1788845044.335635:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:0.0:1788845044.335638:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:0.0:1788845044.335645:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007273a4da reply state 0000000045b010cc transno 4294967845 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.335666:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845044.335793:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641787520:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4943us (5180us total) trans 4294967845 rc 0/0 00000100:00100000:0.0:1788845044.335801:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1550 00000020:00080000:0.0:1788845044.335806:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845044.340003:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.340025:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.340032:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.340039:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02756000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845044.340044:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.340052:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.340068:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.340071:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845044.340076:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.340090:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756000/0xb2187301287aed13 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.341850:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.342095:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641787776 00000100:00100000:0.0:1788845044.342105:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641787776 00000100:00100000:0.0:1788845044.342174:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1551 00000100:00100000:0.0:1788845044.342185:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641787776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.342213:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.342233:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1e7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.342276:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.342289:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.342325:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.342344:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 12 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.342359:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01ced880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660392832:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845044.342443:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.346245:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.346321:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641787904 00000100:00100000:3.0:1788845044.346329:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641787904 00010000:00010000:3.0:1788845044.346368:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 13 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.346409:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 523 00000100:00100000:3.0:1788845044.346420:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d10450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641787904:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845044.346437:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.346463:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 13 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.346481:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 13 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.346503:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.346509:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.346538:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.346549:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.346588:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.346603:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660392960:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845044.346696:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22f00 going to sleep for 11 seconds 00010000:00010000:0.0:1788845044.347275:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287aec80 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa154 expref: 13 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.347292:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01ced880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660392832:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845044.347316:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.347330:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.347333:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.347336:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845044.347380:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641787776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5197us (5539us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.347388:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1551 00000100:00100000:1.0:1788845044.351361:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.351460:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641788032 00000100:00100000:2.0:1788845044.351469:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641788032 00000100:00100000:2.0:1788845044.351524:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1134 00000100:00100000:2.0:1788845044.351546:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+9:14891:x1875739641788032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.351564:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.351582:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1ee expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.351615:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.351628:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 17) 00010000:00010000:2.0:1788845044.351632:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 17) 00010000:00010000:2.0:1788845044.351638:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.351682:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.351699:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 11 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.351714:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660393088:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845044.351793:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.351880:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660392960:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845044.351936:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845044.351965:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d10450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641787904:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5546us (5730us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.351975:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 523 00000100:00100000:1.0:1788845044.355325:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.355410:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641788160 00000100:00100000:3.0:1788845044.355419:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641788160 00010000:00010000:3.0:1788845044.355455:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 12 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.355498:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 524 00000100:00100000:3.0:1788845044.355507:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641788160:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845044.355522:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.355555:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 12 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.355573:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 12 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.355597:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 17) 00010000:00010000:3.0:1788845044.355602:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 17) 00010000:00010000:3.0:1788845044.355610:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.355621:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.355666:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.355680:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660393216:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845044.355756:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00010000:00010000:2.0:1788845044.356199:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e441fd28/0xb2187301287aec5d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1273856->1290239) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa131 expref: 12 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.356216:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660393088:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845044.356241:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.356252:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:2.0:1788845044.356299:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641788032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4755us (4947us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845044.356310:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1134 00000100:00100000:1.0:1788845044.360487:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.360540:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.360644:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660393216:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845044.360680:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845044.360737:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641788160:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5231us (5422us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.360748:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 524 00000100:00100000:0.0:1788845044.360843:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641788288 00000100:00100000:0.0:1788845044.360851:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641788288 00002000:00010000:0.0:1788845044.360925:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845044.360930:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dd880 with x1875739641788288 00000100:00100000:0.0:1788845044.360955:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1001 00000100:00100000:0.0:1788845044.360965:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641788288:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:0.0:1788845044.361247:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962463 -> 650014 00000004:00010000:0.0:1788845044.361252:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:0.0:1788845044.361254:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:0.0:1788845044.361256:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:0.0:1788845044.361259:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845044.361298:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641788288:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 333us (821us total) trans 4294967846 rc 0/0 00000100:00100000:0.0:1788845044.361305:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1001 00002000:00010000:0.0:1788845044.361308:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788845044.361310:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dd880 with x1875739641788288 00000100:00100000:2.0:1788845044.362419:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641788416 00000100:00100000:2.0:1788845044.362430:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641788416 00002000:00010000:2.0:1788845044.362464:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845044.362469:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dc700 with x1875739641788416 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845044.362478:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dc700. 00000100:00100000:2.0:1788845044.362501:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 554 00000100:00100000:2.0:1788845044.362510:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641788416:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845044.363320:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2081862 -> 0 00002000:00010000:2.0:1788845044.363326:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00002000:00010000:2.0:1788845044.363329:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00002000:00010000:2.0:1788845044.363332:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00002000:00010000:2.0:1788845044.363334:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 57 -> 57 00000100:00100000:2.0:1788845044.363393:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641788416:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 883us (2851us total) trans 4294967499 rc 0/0 00000100:00100000:2.0:1788845044.363401:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 554 00002000:00010000:2.0:1788845044.363405:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845044.363408:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f023dc700 with x1875739641788416 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845044.363413:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f023dc700. 00000100:00100000:0.0:1788845044.368212:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.368385:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641788544 00000100:00100000:3.0:1788845044.368397:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641788544 00000100:00100000:3.0:1788845044.368459:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1552 00000100:00100000:3.0:1788845044.368470:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641788544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.368495:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.368517:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1f5 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.368619:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aed2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.368635:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aed2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.368656:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aed2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845044.368684:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 650014 00000004:00200000:3.0:1788845044.368688:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845044.368706:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aed2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.368731:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1f5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.368750:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aed2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1f5 expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.368758:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.368761:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845044.368764:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:3.0:1788845044.368827:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ced880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641788544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 359us (623us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.368834:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1552 00000100:00100000:1.0:1788845044.372839:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.372937:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.373195:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641788800 00000100:00100000:3.0:1788845044.373204:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641788800 00000100:00100000:3.0:1788845044.373259:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1135 00000100:00100000:3.0:1788845044.373269:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641788800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.373290:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.373309:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287aed36 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa203 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.373369:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660393344:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.373445:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.375370:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641788672 00000100:00100000:2.0:1788845044.375383:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641788672 00000100:00100000:2.0:1788845044.375452:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1553 00000100:00100000:2.0:1788845044.375461:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641788672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.375481:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.375500:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aed3d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa1fc expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.375550:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.375589:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660393472:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845044.375671:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.377476:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660393344:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.377515:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287aed36 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa203 expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.377551:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287aed36 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa203 expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.377566:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845044.377624:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641788800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4355us (4685us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.377633:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1135 00000100:00100000:2.0:1788845044.379411:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660393472:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845044.379436:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 650014 00010000:00010000:2.0:1788845044.379454:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aed3d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1fc expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.379473:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.379477:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845044.379482:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aed3d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1fc expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.379496:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:2.0:1788845044.379671:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641788672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4210us (6842us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.379680:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1553 00000100:00100000:0.0:1788845044.384860:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.385066:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.385518:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641789056 00000100:00100000:1.0:1788845044.385540:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641789056 00000100:00100000:1.0:1788845044.385640:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1136 00000100:00100000:1.0:1788845044.385660:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641789056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.385681:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.385700:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aed44 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa211 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.385763:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660393600:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.385850:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.387839:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641788928 00000100:00100000:3.0:1788845044.387852:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641788928 00000100:00100000:0.0:1788845044.388120:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1554 00000100:00100000:0.0:1788845044.388131:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641788928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.388151:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.388170:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aed4b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa20a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.388205:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.388248:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01cee300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660393728:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.388322:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.388921:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660393600:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845044.388952:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aed44 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa211 expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.388973:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aed44 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa211 expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.388986:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757400, rc 0) 00000100:00100000:1.0:1788845044.389018:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641789056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3370us (3953us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.389026:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1136 00000100:00100000:3.0:1788845044.391224:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01cee300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660393728:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845044.391248:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 650014 00010000:00010000:3.0:1788845044.391269:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aed4b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa20a expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.391289:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.391292:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.391297:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287aed4b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa20a expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.391311:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:3.0:1788845044.392234:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641788928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4099us (7378us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.392245:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1554 00000100:00100000:0.0:1788845044.395361:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.395519:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641789184 00000100:00100000:2.0:1788845044.395530:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641789184 00000100:00100000:2.0:1788845044.395688:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1555 00000100:00100000:2.0:1788845044.395698:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641789184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.395718:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.395737:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa218 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.395766:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.395775:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.395806:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.395822:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 19 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.395838:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660393856:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845044.395957:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.398598:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.398725:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660393856:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845044.398760:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.398780:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.398783:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.398788:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.398799:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f800, rc 0) 00000100:00100000:0.0:1788845044.398858:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641789184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3161us (3506us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.398866:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1555 00000100:00100000:3.0:1788845044.399058:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641789312 00000100:00100000:3.0:1788845044.399066:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641789312 00010000:00010000:3.0:1788845044.399096:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 20 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.399121:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 525 00000100:00100000:3.0:1788845044.399131:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641789312:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845044.399143:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.399179:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 20 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.399195:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 20 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.399222:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287aed1a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1e7 expref: 20 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.399235:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.399240:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.399248:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.399271:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.399301:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.399313:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660393984:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845044.399362:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.403802:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.406483:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641789440 00000100:00100000:0.0:1788845044.406501:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641789440 00000100:00100000:0.0:1788845044.406594:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1002 00000100:00100000:0.0:1788845044.406606:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641789440:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845044.406859:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 650014 -> 650014 00000004:00010000:0.0:1788845044.406863:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:0.0:1788845044.406866:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:0.0:1788845044.406868:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:0.0:1788845044.406870:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845044.407429:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641789440:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 824us (3635us total) trans 4294967847 rc 0/0 00000100:00100000:0.0:1788845044.407439:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1002 00000100:00100000:0.0:1788845044.410708:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.410788:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660393984:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845044.410818:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845044.410880:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641789312:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11750us (12292us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.410909:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 525 00000100:00100000:1.0:1788845044.416369:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641789568 00000100:00100000:1.0:1788845044.416385:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641789568 00000100:00100000:1.0:1788845044.416468:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1137 00000100:00100000:1.0:1788845044.416480:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641789568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.416500:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.416531:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aed59 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa21f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.416594:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660394112:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.416702:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.422630:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660394112:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845044.422684:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aed59 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa21f expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.422706:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aed59 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa21f expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.422720:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757400, rc 0) 00000100:00100000:1.0:1788845044.422784:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641789568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6305us (12085us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.422791:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1137 00000100:00100000:0.0:1788845044.425993:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.426126:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641789696 00000100:00100000:3.0:1788845044.426137:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641789696 00000100:00100000:3.0:1788845044.426205:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1138 00000100:00100000:3.0:1788845044.426216:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641789696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.426233:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.426252:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287aed60 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa226 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.426326:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660394240:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.426400:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.429243:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660394240:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.429293:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287aed60 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa226 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.429319:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001685967d/0xb2187301287aed60 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa226 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.429333:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845044.429430:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641789696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3215us (3448us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.429438:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1138 00000100:00100000:0.0:1788845044.438863:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.439079:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641789824 00000100:00100000:3.0:1788845044.439090:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641789824 00000100:00100000:3.0:1788845044.439146:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1556 00000100:00100000:3.0:1788845044.439157:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:14891:x1875739641789824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.439324:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.439349:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa22d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.439385:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.439394:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.439428:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.439446:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 14 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.439460:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660394368:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845044.439669:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.444483:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.444579:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641789952 00000100:00100000:1.0:1788845044.444589:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641789952 00010000:00010000:1.0:1788845044.444633:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 15 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.444688:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 526 00000100:00100000:1.0:1788845044.444709:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641789952:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845044.444726:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845044.444757:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 15 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.444775:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 15 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.444797:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845044.444803:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.444816:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.444825:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.444860:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.444873:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660394496:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845044.444972:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00010000:00010000:3.0:1788845044.448658:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f800/0xb2187301287aed52 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa218 expref: 15 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.448684:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660394368:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845044.448715:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.448734:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.448737:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.448740:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030c00, rc 0) 00000100:00100000:3.0:1788845044.448802:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641789824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9647us (9949us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.448813:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1556 00000100:00100000:1.0:1788845044.455256:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660394496:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845044.455285:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845044.456135:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641789952:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 11423us (11656us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845044.456147:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 526 00000100:00100000:0.0:1788845044.458667:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.458775:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641790080 00000100:00100000:3.0:1788845044.458785:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641790080 00002000:00100000:3.0:1788845044.458838:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036083480 x1875739641790080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.458848:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036083480 with x1875739641790080 00000100:00100000:2.0:1788845044.459814:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1003 00000100:00100000:2.0:1788845044.459832:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2368:x1875739641790080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845044.482840:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:2368:x1875739641790080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 23002us (24168us total) trans 0 rc 651264/651264 00000100:00100000:2.0:1788845044.482856:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1003 00002000:00100000:2.0:1788845044.482870:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a036083480 x1875739641790080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/440 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 651264/651264 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.482880:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036083480 with x1875739641790080 00000100:00100000:2.0:1788845044.487157:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.487356:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641790208 00000100:00100000:1.0:1788845044.487366:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641790208 00000100:00100000:1.0:1788845044.487428:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1557 00000100:00100000:1.0:1788845044.487439:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641790208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.487462:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.487484:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756200/0xb2187301287aed6e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa234 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.487529:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.487569:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660394624:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.487659:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.490914:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.491036:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641790336 00000100:00100000:3.0:1788845044.491046:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641790336 00000100:00100000:3.0:1788845044.491101:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1139 00000100:00100000:3.0:1788845044.491122:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641790336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.491143:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.491161:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287aed75 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa23b expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.491233:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660394752:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.491309:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.491985:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660394624:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845044.492006:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 650014 00010000:00010000:2.0:1788845044.492022:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756200/0xb2187301287aed6e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa234 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.492039:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.492042:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845044.492047:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756200/0xb2187301287aed6e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa234 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.492061:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756200, rc 0) 00000100:00100000:2.0:1788845044.492108:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641790208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4672us (4962us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.492697:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1557 00000100:00100000:3.0:1788845044.496713:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660394752:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.496759:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287aed75 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa23b expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.496785:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b450317d/0xb2187301287aed75 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa23b expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.496801:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030200, rc 0) 00000100:00100000:3.0:1788845044.496864:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641790336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5753us (5990us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.496873:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1139 00000100:00100000:0.0:1788845044.500857:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.500992:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641790464 00000100:00100000:3.0:1788845044.501002:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641790464 00000100:00100000:3.0:1788845044.501051:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1558 00000100:00100000:3.0:1788845044.501060:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641790464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.501081:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.501102:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287aed7c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa242 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.501152:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.501201:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660394880:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.501277:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.504312:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.504476:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660394880:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845044.504499:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 650014 00010000:00010000:3.0:1788845044.504531:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287aed7c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa242 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.504554:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.504557:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.504562:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287aed7c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa242 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.504578:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845044.504637:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641790464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3577us (3788us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.504657:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1558 00000100:00100000:1.0:1788845044.505380:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641790592 00000100:00100000:1.0:1788845044.505388:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641790592 00000100:00100000:1.0:1788845044.505437:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1140 00000100:00100000:1.0:1788845044.505447:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641790592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.505462:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.505477:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287aed83 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa249 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.505539:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660395008:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.505613:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.510729:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660395008:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845044.510770:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287aed83 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa249 expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.510800:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287aed83 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa249 expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.510813:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756000, rc 0) 00000100:00100000:1.0:1788845044.510876:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641790592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5429us (6573us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.510924:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1140 00000100:00100000:0.0:1788845044.518200:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.518621:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641790720 00000100:00100000:2.0:1788845044.518631:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641790720 00000100:00100000:2.0:1788845044.518747:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1559 00000100:00100000:2.0:1788845044.518758:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641790720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.518780:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.518803:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aed8a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa250 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.518929:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.518944:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.518961:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845044.518988:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 650014 00000004:00200000:2.0:1788845044.518992:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845044.519011:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.519037:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aed8a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.519055:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 21 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.519064:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.519067:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845044.519071:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e000, rc 0) 00000100:00100000:2.0:1788845044.519137:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641790720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 380us (948us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845044.519144:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1559 00000100:00100000:0.0:1788845044.522987:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.525057:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.525604:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641790848 00000100:00100000:1.0:1788845044.525616:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641790848 00000100:00100000:1.0:1788845044.525635:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641790976 00000100:00100000:1.0:1788845044.525637:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641790976 00000100:00100000:1.0:1788845044.525698:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1141 00000100:00100000:1.0:1788845044.525707:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641790976:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.525725:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.525744:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287aed98 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa25e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.525802:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660395136:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.525880:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.527436:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1560 00000100:00100000:1.0:1788845044.527447:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641790848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.527465:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.527484:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287aed9f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa257 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.527591:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.527627:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f23100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660395264:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.527691:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00002000:00010000:0.0:1788845044.528695:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845043 -> 1788845044 00000100:00100000:0.0:1788845044.528721:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660395136:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845044.528752:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287aed98 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa25e expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.528774:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a79e525/0xb2187301287aed98 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa25e expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.528787:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756000, rc 0) 00000100:00100000:0.0:1788845044.528840:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641790976:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3133us (3790us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.528848:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1141 00000004:00010000:3.0:1788845044.537714:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845044 00000100:00100000:3.0:1788845044.537752:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f23100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660395264:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845044.537778:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 650014 00010000:00010000:3.0:1788845044.537800:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287aed9f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa257 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.537821:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.537825:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.537830:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287aed9f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa257 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.537847:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757c00, rc 0) 00000100:00100000:3.0:1788845044.537943:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641790848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10497us (14967us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.537952:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1560 00000100:00100000:1.0:1788845044.541257:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.541306:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.541473:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641791232 00000100:00100000:0.0:1788845044.541483:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641791232 00000100:00100000:0.0:1788845044.541542:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1142 00000100:00100000:0.0:1788845044.541552:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641791232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.541572:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.541591:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287aeda6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa26c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.541657:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660395392:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.541732:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.542486:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641791104 00000100:00100000:3.0:1788845044.542493:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641791104 00000100:00100000:3.0:1788845044.542550:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1561 00000100:00100000:3.0:1788845044.542559:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641791104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.542573:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.542588:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287aedad lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa265 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.542620:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.542661:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660395520:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.542714:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.545518:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660395392:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845044.545568:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287aeda6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa26c expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.545596:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287aeda6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa26c expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.545612:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845044.545679:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641791232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4126us (4370us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.545688:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1142 00000100:00100000:3.0:1788845044.549138:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660395520:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845044.549179:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 650014 00010000:00010000:3.0:1788845044.549200:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287aedad lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa265 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.549222:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.549225:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.549229:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030e00/0xb2187301287aedad lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa265 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.549245:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030e00, rc 0) 00000100:00100000:3.0:1788845044.549308:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f23100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641791104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6750us (8059us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.549316:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1561 00000100:00100000:1.0:1788845044.553102:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.553453:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641791488 00000100:00100000:3.0:1788845044.553463:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641791488 00000100:00100000:3.0:1788845044.553562:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1562 00000100:00100000:3.0:1788845044.553573:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641791488:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845044.553627:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.553664:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aed2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1f5 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.553684:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.553689:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845044.553944:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.553959:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.553967:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.553998:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.554017:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 20 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.554032:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660395648:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845044.554120:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.557564:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.557718:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660395648:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845044.557748:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.557975:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641791616 00000100:00100000:2.0:1788845044.557983:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641791616 00010000:00010000:2.0:1788845044.558020:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 21 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.558068:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 527 00000100:00100000:2.0:1788845044.558078:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641791616:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845044.558097:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845044.558104:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 21 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.558122:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 21 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.558145:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e000/0xb2187301287aed91 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa250 expref: 21 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.558158:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.558175:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.558185:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.558193:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.558224:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845044.558462:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.558470:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845044.558603:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967848 00000004:00010000:1.0:1788845044.558671:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 650014 -> 650014 00000004:00010000:1.0:1788845044.558675:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:1.0:1788845044.558678:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845044 -> 1788845043 00000004:00010000:1.0:1788845044.558680:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:1.0:1788845044.558682:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:1.0:1788845044.558689:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000032d53d7c reply state 00000000aa1363bf transno 4294967849 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.558698:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845044.558793:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641791488:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5221us (5699us total) trans 4294967849 rc 0/0 00000100:00100000:1.0:1788845044.558804:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1562 00000020:00080000:1.0:1788845044.558809:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788845044.559565:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641791616:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 1484us (2006us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845044.559576:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 527 00010000:00010000:2.0:1788845044.565979:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.565996:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.566007:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.566013:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02030200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845044.566019:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.566028:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.566046:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.566049:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845044.566054:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.566060:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030200/0xb2187301287aedb4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.568050:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.568141:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641791744 00000100:00100000:1.0:1788845044.568152:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641791744 00000100:00100000:1.0:1788845044.568219:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1563 00000100:00100000:1.0:1788845044.568230:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641791744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.568265:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.568288:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa273 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.568318:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.568328:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.568363:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.568382:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 19 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.568395:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660395776:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845044.568479:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.571824:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.571934:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641791872 00000100:00100000:2.0:1788845044.571943:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641791872 00002000:00100000:2.0:1788845044.571989:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251df80 x1875739641791872/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.572001:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251df80 with x1875739641791872 00010000:00010000:2.0:1788845044.572009:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.572028:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 20 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845044.572041:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0251df80 00000100:00100000:2.0:1788845044.572066:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 37 00000100:00100000:2.0:1788845044.572075:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2370:x1875739641791872:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845044.572712:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660395776:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845044.572744:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.572759:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.572762:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.572766:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.572776:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757800, rc 0) 00000100:00100000:3.0:1788845044.572821:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641791744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4593us (4779us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.572829:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1563 00000004:00010000:2.0:1788845044.576940:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 650014 -> 650014 00000004:00010000:2.0:1788845044.576950:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.576953:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:2.0:1788845044.576955:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.576957:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845044.577026:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+22:2370:x1875739641791872:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4947us (5206us total) trans 4294967850 rc 0/0 00000100:00100000:2.0:1788845044.577038:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 37 00002000:00100000:2.0:1788845044.577052:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0251df80 x1875739641791872/t4294967850(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:150/0 lens 488/448 e 0 to 0 dl 1788845055 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.577062:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0251df80 with x1875739641791872 00010000:00010000:2.0:1788845044.577073:0:5853:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 22 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.577091:0:5853:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 22 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845044.577099:0:5853:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0251df80 00000100:00100000:1.0:1788845044.580663:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.580798:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641792000 00000100:00100000:2.0:1788845044.580808:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641792000 00010000:00010000:2.0:1788845044.580849:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 22 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.580925:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 528 00000100:00100000:2.0:1788845044.580935:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+22:2368:x1875739641792000:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845044.580949:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845044.580975:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 22 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.580992:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 22 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.581013:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030c00/0xb2187301287aed67 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa22d expref: 22 pid: 11036 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.581031:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.581036:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.581047:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.581057:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.581096:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.581114:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660395904:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845044.581980:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.585861:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.586009:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641792128 00000100:00100000:1.0:1788845044.586019:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641792128 00000100:00100000:1.0:1788845044.586078:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1143 00000100:00100000:1.0:1788845044.586091:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641792128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.586112:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.586131:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa27a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.586466:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.586478:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(5 17) 00010000:00010000:1.0:1788845044.586483:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 17) 00010000:00010000:1.0:1788845044.586489:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.586534:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.586553:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 10 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.586568:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02483800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660396032:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845044.586691:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.587261:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660395904:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845044.587282:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845044.587316:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641792000:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6383us (6672us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.587326:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 528 00000100:00100000:0.0:1788845044.594612:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02483800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660396032:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845044.594661:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.594689:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757400, rc 0) 00000100:00100000:0.0:1788845044.594758:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641792128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8669us (8905us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.594767:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1143 00000100:00100000:1.0:1788845044.595098:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.595298:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641792256 00000100:00100000:3.0:1788845044.595306:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641792256 00010000:00010000:3.0:1788845044.595339:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 11 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.595376:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 529 00000100:00100000:3.0:1788845044.595387:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641792256:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845044.595401:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.595425:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 11 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.595444:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 11 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.595468:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aed21 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa1ee expref: 11 pid: 6682 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.595482:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 17) 00010000:00010000:3.0:1788845044.595487:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 17) 00010000:00010000:3.0:1788845044.595494:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.595504:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.595680:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 11 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.595695:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660396160:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845044.595747:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.600283:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.600367:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.600823:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641792512 00000100:00100000:1.0:1788845044.600834:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641792512 00002000:00010000:1.0:1788845044.600880:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845044.600906:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c4a680 with x1875739641792512 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845044.600917:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c4a680. 00000100:00100000:1.0:1788845044.600955:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 555 00000100:00100000:1.0:1788845044.600967:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641792512:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845044.601215:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1788845044.601219:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00002000:00010000:1.0:1788845044.601221:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845044 -> 1788845043 00002000:00010000:1.0:1788845044.601224:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00002000:00010000:1.0:1788845044.601226:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 57 -> 57 00000100:00100000:1.0:1788845044.601295:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641792512:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 330us (1020us total) trans 4294967500 rc 0/0 00000100:00100000:1.0:1788845044.601303:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 555 00002000:00010000:1.0:1788845044.601307:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845044.601309:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c4a680 with x1875739641792512 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845044.601315:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c4a680. 00000100:00100000:2.0:1788845044.602195:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641792384 00000100:00100000:2.0:1788845044.602204:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641792384 00002000:00010000:2.0:1788845044.602238:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845044.602245:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c49c00 with x1875739641792384 00000100:00100000:2.0:1788845044.602290:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1004 00000100:00100000:2.0:1788845044.602299:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641792384:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:2.0:1788845044.602475:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 650014 -> 762699 00000004:00010000:2.0:1788845044.602479:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.602482:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:2.0:1788845044.602484:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.602486:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845044.602563:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641792384:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 264us (2194us total) trans 4294967851 rc 0/0 00000100:00100000:2.0:1788845044.602571:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1004 00002000:00010000:2.0:1788845044.602575:0:13305:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788845044.602577:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01c49c00 with x1875739641792384 00000100:00100000:3.0:1788845044.603081:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660396160:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845044.603106:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845044.603300:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2369:x1875739641792256:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7913us (8202us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845044.603310:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 529 00000100:00100000:1.0:1788845044.606450:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.606636:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641792640 00000100:00100000:2.0:1788845044.606654:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641792640 00000100:00100000:2.0:1788845044.606713:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1564 00000100:00100000:2.0:1788845044.606724:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641792640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.606750:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.606771:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aedc9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa281 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.606858:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.606875:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.606906:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845044.606935:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 762699 00000004:00200000:2.0:1788845044.606939:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845044.606959:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.606987:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f600/0xb2187301287aedc9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.607004:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.607014:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.607017:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845044.607021:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3f400, rc 0) 00000100:00100000:2.0:1788845044.607177:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641792640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 455us (738us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845044.607185:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1564 00000100:00100000:1.0:1788845044.614350:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.614429:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641792896 00000100:00100000:2.0:1788845044.614440:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641792896 00000100:00100000:2.0:1788845044.614501:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1565 00000100:00100000:2.0:1788845044.614511:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641792896:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845044.614820:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.614837:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.614845:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.614879:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.614932:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 17 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.614946:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660396288:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845044.615025:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.618422:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660396288:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845044.618452:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.618566:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.618965:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641793024 00000100:00100000:1.0:1788845044.618974:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641793024 00010000:00010000:1.0:1788845044.619014:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 18 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.619050:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 530 00000100:00100000:1.0:1788845044.619059:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d12c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641793024:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845044.619080:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845044.619088:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 18 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.619106:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 18 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.619132:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3f400/0xb2187301287aedd0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa281 expref: 18 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.619151:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845044.619155:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.619166:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.619174:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.619203:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845044.619268:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d12c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641793024:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 209us (703us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845044.619276:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 530 00010000:00010000:0.0:1788845044.619832:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.619841:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788845044.620011:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967852 00000004:00010000:0.0:1788845044.620066:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 762699 -> 762699 00000004:00010000:0.0:1788845044.620070:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:0.0:1788845044.620072:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:0.0:1788845044.620074:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:0.0:1788845044.620076:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:0.0:1788845044.620083:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000071a880f reply state 00000000e70ab776 transno 4294967853 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.620092:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845044.620187:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641792896:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 5677us (5847us total) trans 4294967853 rc 0/0 00000100:00100000:0.0:1788845044.620195:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1565 00000020:00080000:0.0:1788845044.620199:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788845044.625458:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845044.625690:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.625703:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.625712:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.625717:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031d3fc00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845044.625722:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.625730:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.625747:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.625751:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845044.625755:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.625761:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3fc00/0xb2187301287aedd7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.625839:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641793152 00000100:00100000:2.0:1788845044.625849:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641793152 00000100:00100000:2.0:1788845044.625928:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1566 00000100:00100000:2.0:1788845044.625939:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+23:14891:x1875739641793152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.625960:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.625977:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa288 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.626006:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.626015:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.626046:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.626064:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 16 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.626077:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660396416:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845044.626157:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.629732:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.629801:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641793280 00000100:00100000:0.0:1788845044.629810:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641793280 00010000:00010000:0.0:1788845044.629848:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 17 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.629916:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 531 00000100:00100000:0.0:1788845044.629926:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641793280:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845044.629940:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845044.629968:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 17 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.629984:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 17 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.630005:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.630012:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.630022:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.630030:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.630065:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.630077:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c48380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660396544:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845044.630146:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18600 going to sleep for 11 seconds 00010000:00010000:3.0:1788845044.632390:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aedbb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa273 expref: 17 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.632412:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660396416:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845044.632440:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.632455:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.632458:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.632460:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3e800, rc 0) 00000100:00100000:3.0:1788845044.632559:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641793152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6622us (7111us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.632567:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1566 00000100:00100000:3.0:1788845044.634832:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.635010:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641793408 00000100:00100000:0.0:1788845044.635021:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641793408 00000100:00100000:0.0:1788845044.635083:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1144 00000100:00100000:0.0:1788845044.635094:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+10:14891:x1875739641793408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.635111:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.635132:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa28f expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.635173:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.635184:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(6 17) 00010000:00010000:0.0:1788845044.635188:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 17) 00010000:00010000:0.0:1788845044.635194:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.635227:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 12 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.635246:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 12 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.635260:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660396672:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845044.635332:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.637960:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.638058:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660396672:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845044.638094:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.638118:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845044.638189:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:14891:x1875739641793408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3097us (3364us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.638198:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1144 00000100:00100000:1.0:1788845044.638714:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641793536 00000100:00100000:1.0:1788845044.638721:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641793536 00002000:00100000:1.0:1788845044.638766:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1275220) req@ffff94a03f6c5500 x1875739641793536/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845044.638778:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c5500 with x1875739641793536 ext(1249280->1275220) 00010000:00010000:1.0:1788845044.638785:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 13 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.638805:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 13 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845044.638813:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f6c5500 00000100:00100000:1.0:1788845044.638848:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 33 00000100:00100000:1.0:1788845044.638858:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641793536:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845044.639661:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c48380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660396544:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845044.639680:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845044.639815:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641793280:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9889us (10090us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.639825:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 531 00000100:00100000:2.0:1788845044.643976:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641793536:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5108us (6012us total) trans 4294967501 rc 0/0 00000100:00100000:2.0:1788845044.643989:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 33 00002000:00100000:2.0:1788845044.644003:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1275220) req@ffff94a03f6c5500 x1875739641793536/t4294967501(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/448 e 0 to 0 dl 1788845055 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.644014:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f6c5500 with x1875739641793536 ext(1249280->1275220) 00010000:00010000:2.0:1788845044.644025:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 15 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.644053:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 15 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845044.644062:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f6c5500 00000100:00100000:1.0:1788845044.648996:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.649092:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641793664 00000100:00100000:0.0:1788845044.649104:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641793664 00010000:00010000:0.0:1788845044.649143:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 15 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.649196:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 532 00000100:00100000:0.0:1788845044.649205:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641793664:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845044.649222:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1788845044.649242:0:13304:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1275221 00002000:00010000:0.0:1788845044.649245:0:13304:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845043 -> 1788845044 00010000:00010000:0.0:1788845044.649252:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 15 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.649279:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 15 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.649300:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aedc2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa27a expref: 15 pid: 6684 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.649317:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 17) 00010000:00010000:0.0:1788845044.649322:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 17) 00010000:00010000:0.0:1788845044.649330:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.649340:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.649378:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 10 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.649393:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660396800:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845044.649476:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.652721:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.652764:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.652922:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641793792 00000100:00100000:3.0:1788845044.652931:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641793792 00002000:00010000:3.0:1788845044.652973:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845044.652978:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fbb80 with x1875739641793792 00000100:00100000:3.0:1788845044.653009:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1005 00000100:00100000:3.0:1788845044.653019:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641793792:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845044.653370:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 762699 -> 518024 00000004:00010000:3.0:1788845044.653374:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:3.0:1788845044.653377:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845044.653379:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:3.0:1788845044.653381:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845044.653449:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641793792:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 430us (737us total) trans 4294967854 rc 0/0 00000100:00100000:3.0:1788845044.653458:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1005 00002000:00010000:3.0:1788845044.653461:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845044.653464:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fbb80 with x1875739641793792 00000100:00100000:1.0:1788845044.653684:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641793920 00000100:00100000:1.0:1788845044.653690:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641793920 00002000:00010000:1.0:1788845044.653717:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845044.653721:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0346fb800 with x1875739641793920 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845044.653729:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0346fb800. 00000100:00100000:1.0:1788845044.653760:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 556 00000100:00100000:1.0:1788845044.653768:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641793920:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845044.654013:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1275221 -> 0 00002000:00010000:1.0:1788845044.654017:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00002000:00010000:1.0:1788845044.654020:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845044 -> 1788845044 00002000:00010000:1.0:1788845044.654021:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00002000:00010000:1.0:1788845044.654024:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 57 -> 57 00000100:00100000:1.0:1788845044.654051:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:2369:x1875739641793920:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 284us (1286us total) trans 4294967502 rc 0/0 00000100:00100000:1.0:1788845044.654059:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 556 00002000:00010000:1.0:1788845044.654062:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845044.654065:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0346fb800 with x1875739641793920 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845044.654070:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0346fb800. 00000100:00100000:0.0:1788845044.654801:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660396800:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845044.654822:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845044.654866:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641793664:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5662us (5881us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.654874:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 532 00000100:00100000:2.0:1788845044.660045:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.660151:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641794048 00000100:00100000:3.0:1788845044.660231:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641794048 00000100:00100000:3.0:1788845044.660328:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1567 00000100:00100000:3.0:1788845044.660340:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641794048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.660362:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.660385:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287aedec lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa296 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.660472:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aedf3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.660487:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aedf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.660499:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aedf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845044.660539:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 518024 00000004:00200000:3.0:1788845044.660544:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845044.660565:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aedf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.660593:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031200/0xb2187301287aedec lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa296 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.660612:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aedf3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa296 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.660621:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.660624:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845044.660627:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031c00, rc 0) 00000100:00100000:3.0:1788845044.660698:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641794048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 360us (662us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.660705:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1567 00000100:00100000:2.0:1788845044.663994:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.664088:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641794176 00000100:00100000:0.0:1788845044.664099:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641794176 00000100:00100000:0.0:1788845044.664154:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1568 00000100:00100000:0.0:1788845044.664176:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641794176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.664200:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.664222:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287aedfa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa29d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.664261:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.664308:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660396928:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.664400:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.666455:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.666556:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641794304 00000100:00100000:2.0:1788845044.666566:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641794304 00000100:00100000:2.0:1788845044.666628:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1145 00000100:00100000:2.0:1788845044.666639:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641794304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.666670:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.666688:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aee01 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2a4 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.666743:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660397056:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845044.666819:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000004:00010000:0.0:1788845044.669961:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845044 00000100:00100000:0.0:1788845044.669998:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660396928:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845044.670024:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 518024 00010000:00010000:0.0:1788845044.670046:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287aedfa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa29d expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.670068:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.670071:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.670076:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287aedfa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa29d expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.670092:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845044.670151:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641794176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5989us (6167us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.670159:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1568 00000100:00100000:2.0:1788845044.672877:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660397056:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845044.672965:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aee01 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2a4 expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.672993:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000008715fee/0xb2187301287aee01 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2a4 expref: 15 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.673012:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a031d3fc00, rc 0) 00000100:00100000:2.0:1788845044.673072:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641794304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6435us (6623us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.673080:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1145 00000100:00100000:2.0:1788845044.676467:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.676502:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.676629:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641794560 00000100:00100000:3.0:1788845044.676639:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641794560 00000100:00100000:3.0:1788845044.676716:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1146 00000100:00100000:3.0:1788845044.676727:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641794560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.676749:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.676769:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287aee08 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2b2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.676832:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660397184:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.676932:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.677373:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641794432 00000100:00100000:0.0:1788845044.677380:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641794432 00000100:00100000:0.0:1788845044.677415:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1569 00000100:00100000:0.0:1788845044.677423:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641794432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.677437:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.677451:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee0f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2ab expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.677480:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.677506:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660397312:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.677557:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.681421:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660397312:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845044.681449:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 518024 00010000:00010000:0.0:1788845044.681470:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee0f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ab expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.681492:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.681495:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.681499:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee0f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ab expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.681515:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845044.681597:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641794432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4174us (5092us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.681606:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1569 00000100:00100000:3.0:1788845044.681863:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660397184:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.681924:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287aee08 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b2 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.681942:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a392054/0xb2187301287aee08 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b2 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.681953:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031200, rc 0) 00000100:00100000:3.0:1788845044.681987:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2370:x1875739641794560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5262us (5533us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.681994:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1146 00000100:00100000:2.0:1788845044.686432:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.686505:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641794688 00000100:00100000:3.0:1788845044.686514:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641794688 00000100:00100000:3.0:1788845044.686572:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1570 00000100:00100000:3.0:1788845044.686582:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641794688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.686602:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.686620:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2b9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.686656:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.686663:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b9 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.686691:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.686704:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 24 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.686714:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660397440:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845044.686788:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.689449:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660397440:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845044.689487:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b9 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.689506:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.689509:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.689514:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b9 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.689523:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845044.689624:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641794688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3042us (3203us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.689631:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1570 00000100:00100000:2.0:1788845044.690183:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.690590:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641794816 00000100:00100000:3.0:1788845044.690598:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641794816 00010000:00010000:3.0:1788845044.690631:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 25 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.690706:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 533 00000100:00100000:3.0:1788845044.690717:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641794816:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845044.690730:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.690766:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 25 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.690783:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 25 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.690804:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031d3e800/0xb2187301287aedde lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa288 expref: 25 pid: 5841 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.690818:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.690824:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.690834:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.690843:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.690873:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.690895:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660397568:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845044.690982:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.693108:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660397568:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845044.693134:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845044.693186:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641794816:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2471us (3004us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.693197:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 533 00000100:00100000:1.0:1788845044.697127:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.697393:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641794944 00000100:00100000:2.0:1788845044.697404:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641794944 00002000:00100000:2.0:1788845044.697453:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02360700 x1875739641794944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.697463:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02360700 with x1875739641794944 00000100:00100000:2.0:1788845044.697503:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1006 00000100:00100000:2.0:1788845044.697512:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641794944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845044.722102:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641794944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 24578us (24976us total) trans 0 rc 520192/520192 00000100:00100000:2.0:1788845044.722116:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1006 00002000:00100000:2.0:1788845044.722131:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02360700 x1875739641794944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/440 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 520192/520192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845044.722142:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02360700 with x1875739641794944 00000100:00100000:2.0:1788845044.727388:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.728387:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641795072 00000100:00100000:3.0:1788845044.728400:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641795072 00000100:00100000:3.0:1788845044.728524:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1571 00000100:00100000:3.0:1788845044.728536:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641795072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.728562:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.728584:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287aee1d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2c0 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.728687:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.728705:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.728717:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845044.728810:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 518024 00000004:00200000:3.0:1788845044.728815:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845044.728837:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.728865:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287aee1d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.728945:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.728955:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.728958:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845044.728961:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:3.0:1788845044.729080:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641795072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 546us (1703us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.729088:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1571 00000100:00100000:1.0:1788845044.731956:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.732070:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641795200 00000100:00100000:0.0:1788845044.732080:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641795200 00000100:00100000:0.0:1788845044.732145:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1572 00000100:00100000:0.0:1788845044.732156:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641795200:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.732275:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.732297:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2c7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.732332:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845044.732487:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 518024 00010000:00010000:0.0:1788845044.732499:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.732583:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.732588:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.732593:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845044.733117:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641795200:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 960us (1170us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.733126:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1572 00000100:00100000:0.0:1788845044.738940:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.739154:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641795456 00000100:00100000:2.0:1788845044.739165:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641795456 00000100:00100000:2.0:1788845044.739273:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1573 00000100:00100000:2.0:1788845044.739284:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641795456:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845044.739340:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845044.739369:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031c00/0xb2187301287aedf3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa296 expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.739390:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.739395:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845044.739607:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.739622:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.739632:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.739677:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.739697:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 25 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.739711:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660397696:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845044.739862:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.749428:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660397696:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845044.749461:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.754072:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.757247:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641795584 00000100:00100000:3.0:1788845044.757263:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641795584 00010000:00010000:3.0:1788845044.757312:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 26 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.757363:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 534 00000100:00100000:3.0:1788845044.757373:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2370:x1875739641795584:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845044.757387:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.757395:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 26 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.757413:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 26 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.757434:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee24 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c0 expref: 26 pid: 5840 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.757450:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.757455:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.757464:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.757471:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.757509:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845044.757584:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2370:x1875739641795584:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 212us (3523us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.757592:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 534 00010000:00010000:2.0:1788845044.758077:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.758091:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845044.758214:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967855 00000004:00010000:2.0:1788845044.758265:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 518024 -> 518024 00000004:00010000:2.0:1788845044.758270:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.758273:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845044 -> 1788845043 00000004:00010000:2.0:1788845044.758275:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.758277:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:2.0:1788845044.758287:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004f82dbf2 reply state 0000000014bcd5b4 transno 4294967856 ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.758299:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845044.758392:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:14891:x1875739641795456:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 19109us (19461us total) trans 4294967856 rc 0/0 00000100:00100000:2.0:1788845044.758399:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1573 00000020:00080000:2.0:1788845044.758403:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845044.760248:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.760265:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.760273:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.760280:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a001661e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845044.760285:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.760297:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.760313:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.760316:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845044.760320:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.760325:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a001661e00/0xb2187301287aee32 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.767776:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.767975:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641795712 00000100:00100000:3.0:1788845044.767986:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641795712 00000100:00100000:3.0:1788845044.768052:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1574 00000100:00100000:3.0:1788845044.768063:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:14891:x1875739641795712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.768109:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.768112:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845044.768159:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee16 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2b9 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.768180:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.768185:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845044.768200:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2ce expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.768224:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.768236:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.768271:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.768291:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 24 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.768305:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660397824:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845044.768393:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.772561:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660397824:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845044.772609:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.772631:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.772634:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845044.772638:0:5841:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.772659:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:3.0:1788845044.772726:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:14891:x1875739641795712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4664us (4960us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.772735:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1574 00000100:00100000:2.0:1788845044.773774:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.773930:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641795840 00000100:00100000:0.0:1788845044.773938:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641795840 00010000:00010000:0.0:1788845044.773974:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 25 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.774015:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 535 00000100:00100000:0.0:1788845044.774025:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641795840:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845044.774037:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845044.774063:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 25 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.774078:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 25 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.774098:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aee2b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2c7 expref: 25 pid: 5839 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.774113:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.774118:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.774129:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.774137:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.774170:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.774187:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660397952:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845044.774268:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845044.779499:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.779607:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660397952:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845044.779633:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845044.779714:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03446300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+24:2369:x1875739641795840:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5689us (5943us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845044.779725:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 535 00000100:00100000:2.0:1788845044.779994:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641795968 00000100:00100000:2.0:1788845044.780002:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641795968 00000100:00100000:2.0:1788845044.780052:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1147 00000100:00100000:2.0:1788845044.780062:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641795968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.780080:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.780100:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2d5 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.780132:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.780141:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(7 17) 00010000:00010000:2.0:1788845044.780146:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 17) 00010000:00010000:2.0:1788845044.780150:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.780186:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.780206:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 11 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.780220:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660398080:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845044.780271:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.792250:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660398080:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845044.792305:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.792328:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a001660c00, rc 0) 00000100:00100000:2.0:1788845044.792387:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641795968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12326us (12896us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845044.792396:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1147 00000100:00100000:1.0:1788845044.799148:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.799240:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641796096 00000100:00100000:3.0:1788845044.799252:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641796096 00010000:00010000:3.0:1788845044.799295:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 12 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.799342:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 536 00000100:00100000:3.0:1788845044.799353:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:2369:x1875739641796096:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845044.799368:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845044.799394:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 12 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.799413:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 12 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.799439:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004851d2ef/0xb2187301287aede5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa28f expref: 12 pid: 6685 timeout: 533 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.799457:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 17) 00010000:00010000:3.0:1788845044.799461:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(8 17) 00010000:00010000:3.0:1788845044.799468:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.799477:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.799516:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.799531:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660398208:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845044.799616:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.807774:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660398208:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845044.807803:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845044.807864:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641796096:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8512us (8724us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.807875:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 536 00000100:00100000:1.0:1788845044.819173:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.819239:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.819337:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641796352 00000100:00100000:2.0:1788845044.819348:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641796352 00002000:00010000:2.0:1788845044.819399:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845044.819404:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f0a80 with x1875739641796352 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845044.819417:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f0a80. 00000100:00100000:2.0:1788845044.819447:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 557 00000100:00100000:2.0:1788845044.819457:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641796352:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845044.819688:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1788845044.819693:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00002000:00010000:2.0:1788845044.819704:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845044 -> 1788845044 00002000:00010000:2.0:1788845044.819706:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00002000:00010000:2.0:1788845044.819709:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 57 -> 57 00000100:00100000:2.0:1788845044.819783:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641796352:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 326us (542us total) trans 4294967503 rc 0/0 00000100:00100000:2.0:1788845044.819791:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 557 00002000:00010000:2.0:1788845044.819795:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845044.819798:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f0a80 with x1875739641796352 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845044.819802:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f0a80. 00000100:00100000:3.0:1788845044.821101:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641796224 00000100:00100000:3.0:1788845044.821113:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641796224 00002000:00010000:3.0:1788845044.821155:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845044.821160:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1f80 with x1875739641796224 00000100:00100000:3.0:1788845044.821202:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1007 00000100:00100000:3.0:1788845044.821211:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641796224:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845044.821390:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 518024 -> 685309 00000004:00010000:3.0:1788845044.821393:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:3.0:1788845044.821396:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845044.821398:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:3.0:1788845044.821401:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845044.821527:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641796224:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 317us (2363us total) trans 4294967857 rc 0/0 00000100:00100000:3.0:1788845044.821535:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1007 00002000:00010000:3.0:1788845044.821539:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845044.821542:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1f80 with x1875739641796224 00000100:00100000:1.0:1788845044.830833:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.831163:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641796480 00000100:00100000:3.0:1788845044.831174:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641796480 00000100:00100000:3.0:1788845044.831299:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1575 00000100:00100000:3.0:1788845044.831311:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+24:14891:x1875739641796480:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.831332:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.831353:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee47 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2dc expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.831461:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287aee4e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.831476:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287aee4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.831491:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287aee4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845044.831517:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 685309 00000004:00200000:3.0:1788845044.831522:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845044.831539:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287aee4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.831563:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee47 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2dc expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.831582:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287aee4e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2dc expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.831591:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.831594:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845044.831597:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031400, rc 0) 00000100:00100000:3.0:1788845044.831676:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641796480:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 367us (851us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.831682:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1575 00000100:00100000:1.0:1788845044.836340:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.836390:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.836466:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641796736 00000100:00100000:2.0:1788845044.836476:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641796736 00000100:00100000:2.0:1788845044.836532:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1148 00000100:00100000:2.0:1788845044.836542:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641796736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.836560:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.836578:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d351ac1e/0xb2187301287aee55 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2ea expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.836651:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660398336:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.836684:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641796608 00000100:00100000:0.0:1788845044.836689:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641796608 00000100:00100000:0.0:1788845044.836725:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1576 00000100:00100000:0.0:1788845044.836732:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641796608:12345-192.168.201.29@tcp:101:fsx.0 00000100:00100000:2.0:1788845044.836733:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00010000:00010000:0.0:1788845044.836753:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.836767:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287aee5c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2e3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.836796:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.836817:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660398464:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.836844:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000004:00010000:1.0:1788845044.843728:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845044 00000100:00100000:1.0:1788845044.843767:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c4aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660398464:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845044.843792:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 685309 00010000:00010000:1.0:1788845044.843813:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287aee5c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2e3 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.843835:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845044.843838:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845044.843843:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287aee5c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2e3 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.843864:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:1.0:1788845044.844077:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641796608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7344us (7745us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.844088:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1576 00000100:00100000:2.0:1788845044.846909:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660398336:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845044.846951:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d351ac1e/0xb2187301287aee55 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ea expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.846985:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d351ac1e/0xb2187301287aee55 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ea expref: 12 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.846999:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a001660e00, rc 0) 00000100:00100000:2.0:1788845044.847071:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641796736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10531us (10679us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.847081:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1148 00000100:00100000:0.0:1788845044.860844:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.860937:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641796864 00000100:00100000:0.0:1788845044.860940:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.860950:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641796864 00000100:00100000:2.0:1788845044.861013:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1577 00000100:00100000:2.0:1788845044.861024:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641796864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.861045:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.861068:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aee63 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2f1 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.861107:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.861150:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660398592:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845044.862753:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.867096:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641796992 00000100:00100000:3.0:1788845044.867121:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641796992 00000100:00100000:3.0:1788845044.867396:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1149 00000100:00100000:3.0:1788845044.867409:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2367:x1875739641796992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.867428:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.867447:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287aee6a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2f8 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.867504:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660398720:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.867583:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.871686:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660398720:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.871731:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287aee6a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2f8 expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.871767:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287aee6a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2f8 expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.871782:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845044.871843:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2367:x1875739641796992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4437us (10902us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845044.871852:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1149 00000100:00100000:2.0:1788845044.872506:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660398592:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845044.872523:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 685309 00010000:00010000:2.0:1788845044.872540:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aee63 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2f1 expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.872561:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.872564:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845044.872568:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aee63 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2f1 expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.872580:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a001660e00, rc 0) 00000100:00100000:2.0:1788845044.872627:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641796864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11605us (11801us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.872635:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1577 00000100:00100000:1.0:1788845044.882026:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845044.882954:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641797120 00000100:00100000:2.0:1788845044.882968:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641797120 00000100:00100000:2.0:1788845044.883050:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1008 00000100:00100000:2.0:1788845044.883062:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2368:x1875739641797120:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845044.883440:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 685309 -> 685309 00000004:00010000:2.0:1788845044.883445:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.883448:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845044 -> 1788845043 00000004:00010000:2.0:1788845044.883450:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845044 -> 1788845044 00000004:00010000:2.0:1788845044.883452:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845044.883539:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:2368:x1875739641797120:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 479us (1524us total) trans 4294967858 rc 0/0 00000100:00100000:2.0:1788845044.883547:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1008 00000100:00100000:0.0:1788845044.886668:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.886856:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641797248 00000100:00100000:3.0:1788845044.886867:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641797248 00000100:00100000:3.0:1788845044.886973:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1578 00000100:00100000:3.0:1788845044.886986:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+21:14891:x1875739641797248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.887015:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845044.887039:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee71 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa2ff expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.887147:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.887165:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.887177:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845044.887206:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 685309 00000004:00200000:3.0:1788845044.887211:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845044.887231:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.887261:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aee71 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 23 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.887284:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845044.887294:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845044.887298:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845044.887301:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030400, rc 0) 00000100:00100000:3.0:1788845044.887364:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:14891:x1875739641797248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 380us (707us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845044.887371:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1578 00000100:00100000:0.0:1788845044.899078:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.899176:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641797376 00000100:00100000:3.0:1788845044.899186:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641797376 00002000:00100000:3.0:1788845044.899240:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f827800 x1875739641797376/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 504/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845044.899253:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f827800 with x1875739641797376 00000100:00100000:3.0:1788845044.899295:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1009 00000100:00100000:3.0:1788845044.899306:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2369:x1875739641797376:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845044.914673:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2369:x1875739641797376:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 15355us (15592us total) trans 0 rc 172032/172032 00000100:00100000:1.0:1788845044.914691:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1009 00002000:00100000:1.0:1788845044.914710:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f827800 x1875739641797376/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 504/440 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 172032/172032 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845044.914722:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f827800 with x1875739641797376 00000100:00100000:3.0:1788845044.920114:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.920219:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641797504 00000100:00100000:0.0:1788845044.920230:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641797504 00002000:00100000:0.0:1788845044.920281:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c7800 x1875739641797504/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/0 e 0 to 0 dl 1788845055 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845044.920293:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c7800 with x1875739641797504 00000100:00100000:0.0:1788845044.920337:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1010 00000100:00100000:0.0:1788845044.920346:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641797504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845044.920627:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641797504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (523us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845044.920635:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1010 00002000:00100000:0.0:1788845044.920651:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c7800 x1875739641797504/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:150/0 lens 488/4536 e 0 to 0 dl 1788845055 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845044.920660:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c7800 with x1875739641797504 00000100:00100000:0.0:1788845044.927778:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.927841:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641797632 00000100:00100000:1.0:1788845044.927855:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641797632 00000100:00100000:1.0:1788845044.928027:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1579 00000100:00100000:1.0:1788845044.928038:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641797632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.928059:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.928080:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287aee7f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa306 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.928130:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.928181:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660398848:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.928272:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.935602:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.935717:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641797760 00000100:00100000:0.0:1788845044.935731:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641797760 00000100:00100000:0.0:1788845044.935804:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1150 00000100:00100000:0.0:1788845044.935814:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641797760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.935833:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.935854:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aee86 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa30d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845044.935972:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f827480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660398976:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.936074:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18480 going to sleep for 11 seconds 00000004:00010000:1.0:1788845044.936127:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845044 00000100:00100000:1.0:1788845044.936156:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660398848:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845044.936191:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 685309 00010000:00010000:1.0:1788845044.936210:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287aee7f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa306 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.936231:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845044.936234:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845044.936240:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02756a00/0xb2187301287aee7f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa306 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.936255:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756a00, rc 0) 00000100:00100000:1.0:1788845044.936288:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641797632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8252us (8521us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.936295:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1579 00000100:00100000:0.0:1788845044.941756:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f827480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660398976:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845044.941809:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aee86 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa30d expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.941841:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cdfdfd78/0xb2187301287aee86 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa30d expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.941859:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845044.941979:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641797760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6165us (6388us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.941990:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1150 00000100:00100000:3.0:1788845044.945849:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.945908:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.945976:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641798016 00000100:00100000:1.0:1788845044.945988:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641798016 00000100:00100000:1.0:1788845044.946049:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1151 00000100:00100000:1.0:1788845044.946060:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641798016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845044.946080:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845044.946097:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b34d57a8/0xb2187301287aee8d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa31b expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845044.946158:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660399104:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845044.946255:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.946377:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641797888 00000100:00100000:2.0:1788845044.946382:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641797888 00000100:00100000:2.0:1788845044.946464:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1580 00000100:00100000:2.0:1788845044.946474:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641797888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.946486:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845044.946501:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aee94 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa314 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.946534:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.946560:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660399232:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845044.946609:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.955115:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660399232:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845044.955146:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 685309 00010000:00010000:2.0:1788845044.955169:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aee94 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa314 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.955202:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.955205:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845044.955211:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aee94 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa314 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.955228:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a001660e00, rc 0) 00000100:00100000:2.0:1788845044.955292:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641797888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8820us (9453us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.955301:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1580 00000100:00100000:1.0:1788845044.956977:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660399104:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845044.957017:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b34d57a8/0xb2187301287aee8d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa31b expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.957044:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b34d57a8/0xb2187301287aee8d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa31b expref: 12 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845044.957057:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756a00, rc 0) 00000100:00100000:1.0:1788845044.957132:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641798016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11074us (11222us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845044.957142:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1151 00000100:00100000:3.0:1788845044.961974:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845044.962017:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.962126:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641798272 00000100:00100000:0.0:1788845044.962137:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641798272 00000100:00100000:3.0:1788845044.962367:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1152 00000100:00100000:3.0:1788845044.962382:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641798272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.962401:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1788845044.962401:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641798144 00000100:00100000:2.0:1788845044.962407:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641798144 00010000:00010000:3.0:1788845044.962421:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287aee9b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa329 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.962455:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1581 00000100:00100000:2.0:1788845044.962465:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641798144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845044.962478:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1788845044.962487:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660399360:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845044.962492:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aeea2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa322 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.962527:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845044.962555:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660399488:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845044.962577:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.962590:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845044.965744:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660399488:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845044.965775:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 685309 00010000:00010000:2.0:1788845044.965798:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aeea2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa322 expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845044.965804:0:6682:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1117133 00000100:00100000:0.0:1788845044.965825:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660399360:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845044.965826:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845044.965833:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845044.965840:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a001660e00/0xb2187301287aeea2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa322 expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.965855:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287aee9b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa329 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.965856:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a001660e00, rc 0) 00010000:00010000:0.0:1788845044.965876:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009268add5/0xb2187301287aee9b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa329 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.965909:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030600, rc 0) 00000100:00100000:2.0:1788845044.965993:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641798144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3481us (3979us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.966003:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1581 00000100:00100000:0.0:1788845044.967992:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:2368:x1875739641798272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5604us (5963us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.968005:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1152 00000100:00100000:1.0:1788845044.971463:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845044.971514:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845044.971602:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641798400 00000100:00100000:0.0:1788845044.971612:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641798400 00000100:00100000:0.0:1788845044.971687:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1582 00000100:00100000:0.0:1788845044.971698:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641798400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845044.971718:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845044.971740:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287aeea9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa330 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.971743:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641798528 00000100:00100000:3.0:1788845044.971750:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641798528 00010000:00010000:0.0:1788845044.971780:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.971795:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1153 00000100:00100000:3.0:1788845044.971805:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641798528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845044.971816:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1788845044.971826:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f827480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660399616:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845044.971830:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287aeeb0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa337 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845044.971876:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660399744:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845044.971936:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18480 going to sleep for 11 seconds 00000100:00100000:3.0:1788845044.971943:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845044.995285:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f827480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660399616:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845044.995313:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 685309 00010000:00010000:0.0:1788845044.995331:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287aeea9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa330 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.995350:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845044.995353:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845044.995358:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287aeea9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa330 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845044.995382:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845044.995447:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641798400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 23750us (23996us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845044.995456:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1582 00000100:00100000:2.0:1788845044.995841:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660399744:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845044.995864:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287aeeb0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa337 expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.995931:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ea1513a/0xb2187301287aeeb0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa337 expref: 12 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845044.995942:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:2.0:1788845044.995971:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+11:2369:x1875739641798528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 24169us (24457us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845044.995979:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1153 00000100:00100000:3.0:1788845045.007250:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.007763:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641798656 00000100:00100000:0.0:1788845045.007773:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641798656 00000100:00100000:0.0:1788845045.007845:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1154 00000100:00100000:0.0:1788845045.007858:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+11:14891:x1875739641798656:12345-192.168.201.29@tcp:101:fsx.0 00000100:00080000:1.0:1788845045.007871:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1788845045.007878:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.007946:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa33e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.007986:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.007995:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(8 17) 00010000:00010000:1.0:1788845045.008000:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 17) 00010000:00010000:1.0:1788845045.008006:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [2072576->2097151] (req 2072576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.008044:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.008067:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 15 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.008082:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660399872:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845045.008196:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.011387:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.011505:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641798784 00000100:00100000:1.0:1788845045.011517:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641798784 00002000:00100000:1.0:1788845045.011602:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1105920->1117132) req@ffff94a03532b480 x1875739641798784/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 11704/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.011614:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03532b480 with x1875739641798784 ext(1105920->1117132) 00010000:00010000:1.0:1788845045.011627:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 16 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.011666:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 16 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845045.011676:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03532b480 00000100:00100000:1.0:1788845045.011707:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 34 00000100:00100000:1.0:1788845045.011719:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641798784:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845045.012285:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641798784:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 565us (907us total) trans 4294967504 rc 0/0 00000100:00100000:1.0:1788845045.012295:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 34 00002000:00100000:1.0:1788845045.012304:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1105920->1117132) req@ffff94a03532b480 x1875739641798784/t4294967504(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 11704/448 e 0 to 0 dl 1788845056 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.012314:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03532b480 with x1875739641798784 ext(1105920->1117132) 00010000:00010000:1.0:1788845045.012320:0:6688:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 18 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.012336:0:6688:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 18 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845045.012344:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03532b480 00000100:00080000:3.0:1788845045.012604:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845045.012974:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660399872:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845045.013005:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [2072576->2097151] (req 2072576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 12 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.013022:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756e00, rc 0) 00000100:00100000:2.0:1788845045.013060:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:14891:x1875739641798656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5206us (5821us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845045.013068:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1154 00000100:00100000:1.0:1788845045.016716:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.017028:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641798912 00000100:00100000:3.0:1788845045.017041:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641798912 00010000:00010000:3.0:1788845045.017096:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 18 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.017151:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 537 00000100:00100000:3.0:1788845045.017162:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d13050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641798912:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845045.017176:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845045.017210:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 18 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.017231:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 18 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.017265:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000520948da/0xb2187301287aee40 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2d5 expref: 18 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.017286:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 17) 00010000:00010000:3.0:1788845045.017291:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 17) 00010000:00010000:3.0:1788845045.017300:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.017311:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.017351:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.017372:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660400000:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845045.017457:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.021605:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.021719:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641799040 00000100:00100000:1.0:1788845045.021729:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641799040 00000100:00100000:1.0:1788845045.021789:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1155 00000100:00100000:1.0:1788845045.021799:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641799040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.021819:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.021841:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeebe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa345 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.021934:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660400128:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.022013:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.022495:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660400000:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845045.022528:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845045.022608:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d13050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641798912:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5447us (5904us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845045.022619:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 537 00002000:00010000:1.0:1788845045.029621:0:6684:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 1117133 -> 2097152 00002000:00010000:1.0:1788845045.029632:0:6684:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845044 -> 1788845045 00002000:00010000:1.0:1788845045.029645:0:6684:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845044 -> 1788845045 00000100:00100000:1.0:1788845045.029674:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660400128:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845045.029712:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeebe lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa345 expref: 18 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.029739:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeebe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa345 expref: 18 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.029753:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757000, rc 0) 00000100:00100000:1.0:1788845045.029816:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641799040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8018us (8222us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845045.029823:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1155 00000100:00100000:0.0:1788845045.041585:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.046702:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641799168 00000100:00100000:3.0:1788845045.046719:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641799168 00000100:00100000:3.0:1788845045.046849:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1156 00000100:00100000:3.0:1788845045.046861:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641799168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.046880:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.046928:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeec5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa34c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.046991:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660400256:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.047090:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.050757:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660400256:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845045.050804:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeec5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa34c expref: 18 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.050833:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeec5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa34c expref: 18 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.050856:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:3.0:1788845045.050958:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641799168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4099us (9385us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.050968:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1156 00000100:00100000:0.0:1788845045.055804:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.056086:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641799296 00000100:00100000:2.0:1788845045.056098:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641799296 00000100:00100000:2.0:1788845045.056213:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1157 00000100:00100000:2.0:1788845045.056225:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641799296:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.056243:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.056261:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000044fbafc5/0xb2187301287aeecc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa353 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.056327:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660400384:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.056411:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.063822:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660400384:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845045.063867:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000044fbafc5/0xb2187301287aeecc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa353 expref: 18 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.063928:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000044fbafc5/0xb2187301287aeecc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa353 expref: 18 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.063942:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a001661a00, rc 0) 00000100:00100000:3.0:1788845045.063995:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641799296:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7772us (8203us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.064004:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1157 00000100:00100000:0.0:1788845045.073582:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.073740:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641799424 00000100:00100000:2.0:1788845045.073752:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641799424 00000100:00100000:2.0:1788845045.073813:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1158 00000100:00100000:2.0:1788845045.073823:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641799424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.073842:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.073861:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000fb01089/0xb2187301287aeed3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa35a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.073960:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660400512:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.074044:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.080851:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660400512:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845045.080988:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000fb01089/0xb2187301287aeed3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa35a expref: 18 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.081015:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000fb01089/0xb2187301287aeed3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa35a expref: 18 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.081031:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a001660600, rc 0) 00000100:00100000:3.0:1788845045.085735:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2369:x1875739641799424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11899us (12152us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.085749:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1158 00000100:00100000:0.0:1788845045.091653:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.091757:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641799552 00000100:00100000:1.0:1788845045.091768:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641799552 00000100:00100000:1.0:1788845045.091829:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1159 00000100:00100000:1.0:1788845045.091840:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641799552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.091860:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.091877:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeeda lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa361 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.091975:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660400640:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.092055:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.094723:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660400640:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845045.094767:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeeda lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa361 expref: 18 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.094792:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeeda lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa361 expref: 18 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.094812:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757000, rc 0) 00000100:00100000:1.0:1788845045.094871:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2370:x1875739641799552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3032us (3238us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845045.094880:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1159 00000100:00100000:0.0:1788845045.099345:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.100439:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641799680 00000100:00100000:3.0:1788845045.100457:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641799680 00000100:00100000:3.0:1788845045.100538:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1160 00000100:00100000:3.0:1788845045.100550:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641799680:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.100571:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.100593:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeee1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa368 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.100672:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660400768:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.100766:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.105845:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660400768:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845045.105934:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeee1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa368 expref: 18 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.105960:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeee1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa368 expref: 18 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.105974:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:3.0:1788845045.106024:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641799680:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5476us (6692us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.106033:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1160 00000100:00100000:0.0:1788845045.112192:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.112339:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641799808 00000100:00100000:3.0:1788845045.112349:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641799808 00000100:00100000:3.0:1788845045.112407:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1161 00000100:00100000:3.0:1788845045.112416:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641799808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.112434:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.112451:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeee8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa36f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.112507:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660400896:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.112588:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.115535:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660400896:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845045.115575:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeee8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa36f expref: 18 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.115603:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeee8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa36f expref: 18 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.115618:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:3.0:1788845045.115687:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641799808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3271us (3504us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.115696:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1161 00000100:00100000:0.0:1788845045.120177:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.120275:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641799936 00000100:00100000:1.0:1788845045.120285:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641799936 00000100:00100000:1.0:1788845045.120340:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1162 00000100:00100000:1.0:1788845045.120350:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641799936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.120366:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.120382:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeeef lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa376 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.120435:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660401024:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.120512:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.122667:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660401024:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845045.122709:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeeef lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa376 expref: 18 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.122747:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeeef lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa376 expref: 18 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.122762:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757000, rc 0) 00000100:00100000:1.0:1788845045.122821:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:2368:x1875739641799936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2472us (2654us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845045.122830:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1162 00000100:00100000:0.0:1788845045.130656:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.130753:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641800064 00000100:00100000:1.0:1788845045.130763:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641800064 00000100:00100000:1.0:1788845045.130817:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1163 00000100:00100000:1.0:1788845045.130827:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+17:14891:x1875739641800064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.130846:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.130864:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa37d expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.130925:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.130935:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(9 17) 00010000:00010000:1.0:1788845045.130940:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 17) 00010000:00010000:1.0:1788845045.130946:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1347584->1376255] (req 1347584->1376255) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 18 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.130978:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 11 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.130996:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 11 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.131010:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660401152:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845045.131089:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.134251:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.134363:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641800192 00000100:00100000:2.0:1788845045.134373:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641800192 00002000:00100000:2.0:1788845045.134421:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (2072576->2097151) req@ffff949f02381c00 x1875739641800192/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.134439:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02381c00 with x1875739641800192 ext(2072576->2097151) 00010000:00010000:2.0:1788845045.134448:0:13321:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 12 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.134468:0:13321:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 12 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845045.134477:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f02381c00 00000100:00100000:2.0:1788845045.134517:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 35 00000100:00100000:2.0:1788845045.134526:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+12:2370:x1875739641800192:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845045.134749:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660401152:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845045.134780:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1347584->1376255] (req 1347584->1376255) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 18 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.134796:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757000, rc 0) 00000100:00100000:1.0:1788845045.134824:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:14891:x1875739641800064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3999us (4190us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845045.134831:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1163 00000100:00100000:2.0:1788845045.151491:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641800192:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 16953us (17236us total) trans 4294967505 rc 0/0 00000100:00100000:2.0:1788845045.151507:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 35 00002000:00100000:2.0:1788845045.151531:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (2072576->2097151) req@ffff949f02381c00 x1875739641800192/t4294967505(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/448 e 0 to 0 dl 1788845056 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.151542:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02381c00 with x1875739641800192 ext(2072576->2097151) 00010000:00010000:2.0:1788845045.151553:0:13321:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 14 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.151578:0:13321:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 14 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845045.151586:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f02381c00 00000100:00100000:1.0:1788845045.154754:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.154856:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641800320 00000100:00100000:0.0:1788845045.154865:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641800320 00010000:00010000:0.0:1788845045.154936:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 14 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.154980:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 538 00000100:00100000:0.0:1788845045.154990:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641800320:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845045.155006:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845045.155028:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 14 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.155046:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 14 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.155073:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287aeeb7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2072576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa33e expref: 14 pid: 6685 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.155090:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 17) 00010000:00010000:0.0:1788845045.155094:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 17) 00010000:00010000:0.0:1788845045.155101:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 17 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.155110:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 17 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.155149:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 17 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.155367:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660401280:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845045.155552:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18480 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.159241:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660401280:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845045.159353:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845045.159416:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641800320:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4427us (4671us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845045.159428:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 538 00000100:00100000:0.0:1788845045.163395:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.165417:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641800448 00000100:00100000:1.0:1788845045.165433:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641800448 00002000:00100000:1.0:1788845045.165490:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2097151) req@ffff94a035742300 x1875739641800448/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.165501:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035742300 with x1875739641800448 ext(1048576->2097151) 00002000:00010000:1.0:1788845045.165506:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035742300 00000100:00100000:1.0:1788845045.165606:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 558 00000100:00100000:1.0:1788845045.165616:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641800448:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.174018:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.174250:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641800576 00000100:00100000:0.0:1788845045.174260:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641800576 00002000:00100000:0.0:1788845045.174323:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654000 x1875739641800576/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.174334:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654000 with x1875739641800576 00000100:00100000:0.0:1788845045.174421:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1011 00000100:00100000:0.0:1788845045.174432:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641800576:12345-192.168.201.29@tcp:3:fsx.0 00000100:00080000:3.0:1788845045.174997:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845045.179845:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2370:x1875739641800576:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 5404us (5826us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845045.179860:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1011 00002000:00100000:0.0:1788845045.179874:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f654000 x1875739641800576/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/440 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.179922:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f654000 with x1875739641800576 00000100:00100000:2.0:1788845045.190426:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641800448:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 24801us (27030us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1788845045.190442:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 558 00002000:00100000:2.0:1788845045.190454:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2097151) req@ffff94a035742300 x1875739641800448/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/440 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.190466:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035742300 with x1875739641800448 ext(1048576->2097151) 00002000:00010000:2.0:1788845045.190473:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035742300 00000100:00100000:2.0:1788845045.195064:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.195272:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.195373:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641800832 00000100:00100000:3.0:1788845045.195383:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641800832 00000100:00100000:3.0:1788845045.195442:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1164 00000100:00100000:3.0:1788845045.195451:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641800832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.195471:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.195489:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeefd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa38b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.195564:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660401408:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.195665:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.197339:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641800704 00000100:00100000:1.0:1788845045.197352:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641800704 00000100:00100000:1.0:1788845045.197476:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1583 00000100:00100000:1.0:1788845045.197487:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641800704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.197506:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.197539:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aef04 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa384 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.197577:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.197617:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660401536:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.197710:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00080000:3.0:1788845045.200069:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845045.200466:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660401408:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845045.200504:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeefd lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa38b expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.200535:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bcc6a821/0xb2187301287aeefd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa38b expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.200550:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:3.0:1788845045.200600:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641800832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5150us (5547us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.200609:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1164 00000004:00010000:1.0:1788845045.202011:0:5840:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845044 -> 1788845045 00000004:00010000:1.0:1788845045.202018:0:5840:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845044 -> 1788845045 00000100:00100000:1.0:1788845045.202040:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660401536:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845045.202058:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 685309 00010000:00010000:1.0:1788845045.202073:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aef04 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa384 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.202247:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845045.202251:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845045.202367:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aef04 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa384 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.202381:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757e00, rc 0) 00000100:00100000:1.0:1788845045.202473:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641800704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4989us (7200us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845045.202482:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1583 00000100:00100000:2.0:1788845045.206610:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.206661:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.206768:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641800960 00000100:00100000:3.0:1788845045.206777:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641800960 00000100:00100000:3.0:1788845045.206837:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1584 00000100:00100000:3.0:1788845045.206847:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641800960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.206868:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.206921:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aef0b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa392 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.206963:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.207010:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660401664:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.208130:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.209933:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660401664:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.209963:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 685309 00010000:00010000:2.0:1788845045.209983:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aef0b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa392 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.210005:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.210008:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.210012:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031000/0xb2187301287aef0b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa392 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.210027:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031000, rc 0) 00000100:00100000:2.0:1788845045.211673:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641800960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3234us (3479us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.211695:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1584 00000100:00100000:0.0:1788845045.214559:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641801088 00000100:00100000:0.0:1788845045.214575:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641801088 00000100:00100000:0.0:1788845045.214656:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1165 00000100:00100000:0.0:1788845045.214668:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2367:x1875739641801088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845045.214686:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845045.214703:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287aef12 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa399 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.214763:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035740e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660401792:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845045.214862:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.217628:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035740e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660401792:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845045.217678:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287aef12 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa399 expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.217704:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287aef12 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa399 expref: 14 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.217719:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:0.0:1788845045.217824:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:2367:x1875739641801088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3156us (11160us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845045.217836:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1165 00000100:00100000:3.0:1788845045.225481:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.225767:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641801216 00000100:00100000:2.0:1788845045.225778:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641801216 00000100:00100000:2.0:1788845045.225842:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1585 00000100:00100000:2.0:1788845045.225852:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2370:x1875739641801216:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.225873:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.225913:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287aef19 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3a0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.225946:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.225986:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660401920:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.226064:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.228230:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000004:00010000:3.0:1788845045.228461:0:11036:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 685309 -> 1011804 00000100:00100000:3.0:1788845045.228486:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660401920:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845045.228519:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:3.0:1788845045.228539:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287aef19 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3a0 expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.228559:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845045.228563:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845045.228568:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287aef19 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3a0 expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.228582:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772c00, rc 0) 00000100:00100000:3.0:1788845045.228648:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2370:x1875739641801216:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2797us (3177us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.228656:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1585 00000100:00100000:2.0:1788845045.228834:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641801344 00000100:00100000:2.0:1788845045.228841:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641801344 00000100:00100000:2.0:1788845045.228919:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1166 00000100:00100000:2.0:1788845045.228929:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641801344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.228946:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.228962:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aef20 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3a7 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.229016:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660402048:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.229064:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.232382:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660402048:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845045.232418:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aef20 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3a7 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.232448:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aef20 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3a7 expref: 14 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.232463:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773000, rc 0) 00000100:00100000:2.0:1788845045.232533:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641801344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3605us (4311us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.232542:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1166 00000100:00100000:0.0:1788845045.235805:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.235917:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641801472 00000100:00100000:1.0:1788845045.235927:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641801472 00000100:00100000:1.0:1788845045.235981:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1586 00000100:00100000:1.0:1788845045.235993:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2370:x1875739641801472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.236014:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.236036:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aef27 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3ae expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.236079:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.236120:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660402176:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.236479:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.239596:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.239810:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660402176:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.239835:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845045.239855:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aef27 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ae expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.239876:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.239879:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.239915:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287aef27 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ae expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.239931:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757e00, rc 0) 00000100:00100000:2.0:1788845045.239981:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2370:x1875739641801472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3991us (4185us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.239990:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1586 00000100:00100000:1.0:1788845045.240305:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641801600 00000100:00100000:1.0:1788845045.240313:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641801600 00000100:00100000:1.0:1788845045.240363:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1167 00000100:00100000:1.0:1788845045.240372:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641801600:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.240386:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.240404:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287aef2e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3b5 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.240451:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660402304:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.240513:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.244710:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0351f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660402304:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845045.244750:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287aef2e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3b5 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.244779:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3c1f247/0xb2187301287aef2e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3b5 expref: 14 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.244792:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757a00, rc 0) 00000100:00100000:1.0:1788845045.244854:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641801600:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4482us (5267us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845045.244863:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1167 00000100:00100000:3.0:1788845045.248578:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.248630:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.248731:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641801856 00000100:00100000:0.0:1788845045.248751:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641801856 00000100:00100000:0.0:1788845045.248809:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1168 00000100:00100000:0.0:1788845045.248819:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641801856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845045.248838:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845045.248858:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef35 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3c3 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.248951:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660402432:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845045.249025:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.249356:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641801728 00000100:00100000:2.0:1788845045.249361:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641801728 00000100:00100000:2.0:1788845045.249408:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1587 00000100:00100000:2.0:1788845045.249416:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641801728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.249427:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.249441:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287aef3c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3bc expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.249470:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.249497:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660402560:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.249546:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.257922:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660402560:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.257947:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845045.257963:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287aef3c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3bc expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.257980:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.257982:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.257986:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287aef3c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3bc expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.257997:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773000, rc 0) 00000100:00100000:2.0:1788845045.258043:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2369:x1875739641801728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8628us (9412us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.258049:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1587 00000100:00100000:0.0:1788845045.258327:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660402432:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845045.258348:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef35 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3c3 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.258364:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef35 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3c3 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.258373:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845045.258493:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2369:x1875739641801856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9675us (9923us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845045.258503:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1168 00000100:00100000:3.0:1788845045.265288:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.265391:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641801984 00000100:00100000:0.0:1788845045.265402:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641801984 00000100:00100000:0.0:1788845045.265460:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1169 00000100:00100000:0.0:1788845045.265471:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641801984:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845045.265490:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845045.265508:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3ca expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.265538:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 17 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.265547:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(10 17) 00010000:00010000:0.0:1788845045.265552:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:0.0:1788845045.265557:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1773568->1777663] (req 1773568->1777663) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 14 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.265588:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 17 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.265605:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 17 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.265620:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660402688:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845045.265713:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.269135:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.269527:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035743800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660402688:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845045.269527:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641802112 00000100:00100000:1.0:1788845045.269534:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641802112 00010000:00010000:3.0:1788845045.269563:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1773568->1777663] (req 1773568->1777663) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 14 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:1.0:1788845045.269579:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1347584->1376255) req@ffff949f033f0e00 x1875739641802112/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:3.0:1788845045.269584:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00002000:00100000:1.0:1788845045.269588:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f0e00 with x1875739641802112 ext(1347584->1376255) 00010000:00010000:1.0:1788845045.269594:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 18 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.269610:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 18 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845045.269617:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f033f0e00 00000100:00100000:3.0:1788845045.269652:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641801984:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4182us (4373us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845045.269660:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1169 00000100:00100000:1.0:1788845045.269849:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 36 00000100:00100000:1.0:1788845045.269859:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2369:x1875739641802112:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845045.279817:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641802112:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 9950us (10740us total) trans 4294967506 rc 0/0 00000100:00100000:1.0:1788845045.279835:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 36 00002000:00100000:1.0:1788845045.279849:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1347584->1376255) req@ffff949f033f0e00 x1875739641802112/t4294967506(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/448 e 0 to 0 dl 1788845056 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.279859:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f0e00 with x1875739641802112 ext(1347584->1376255) 00010000:00010000:1.0:1788845045.279869:0:6689:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 20 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.279929:0:6689:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 20 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845045.279938:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff949f033f0e00 00000100:00100000:2.0:1788845045.284235:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.284411:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641802240 00000100:00100000:0.0:1788845045.284422:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641802240 00010000:00010000:0.0:1788845045.284462:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 20 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.284505:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:0.0:1788845045.284526:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2369:x1875739641802240:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845045.284541:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845045.284564:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 20 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.284583:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 20 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.284606:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000ec21ce4/0xb2187301287aeef6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1347584->1376255) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa37d expref: 20 pid: 6684 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.284624:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:0.0:1788845045.284628:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:0.0:1788845045.284647:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.284657:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.284697:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 13 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.284712:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660402816:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845045.284795:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c189c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.291626:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.291728:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641802368 00000100:00100000:3.0:1788845045.291739:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641802368 00002000:00100000:3.0:1788845045.291796:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1773568->1777663) req@ffff949f03457800 x1875739641802368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.291809:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03457800 with x1875739641802368 ext(1773568->1777663) 00002000:00010000:3.0:1788845045.291815:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03457800 00000100:00100000:3.0:1788845045.291853:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 559 00000100:00100000:3.0:1788845045.291863:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641802368:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.292130:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641802368:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (514us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.292139:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 559 00002000:00100000:3.0:1788845045.292145:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1773568->1777663) req@ffff949f03457800 x1875739641802368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.292165:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03457800 with x1875739641802368 ext(1773568->1777663) 00002000:00010000:3.0:1788845045.292169:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03457800 00000100:00100000:2.0:1788845045.292724:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660402816:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845045.292746:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845045.292785:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2369:x1875739641802240:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 8262us (8561us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845045.292794:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 539 00000100:00100000:0.0:1788845045.295966:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.296103:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641802496 00000100:00100000:2.0:1788845045.296113:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641802496 00002000:00100000:2.0:1788845045.296176:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1777664->1781759) req@ffff949f03455c00 x1875739641802496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.296188:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03455c00 with x1875739641802496 ext(1777664->1781759) 00002000:00010000:2.0:1788845045.296194:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03455c00 00000100:00100000:2.0:1788845045.296227:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 560 00000100:00100000:2.0:1788845045.296238:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641802496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.296469:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641802496:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 232us (513us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.296477:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 560 00002000:00100000:2.0:1788845045.296484:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1777664->1781759) req@ffff949f03455c00 x1875739641802496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.296494:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03455c00 with x1875739641802496 ext(1777664->1781759) 00002000:00010000:2.0:1788845045.296498:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03455c00 00000100:00100000:1.0:1788845045.299764:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.300066:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641802624 00000100:00100000:3.0:1788845045.300077:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641802624 00002000:00100000:3.0:1788845045.300128:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1781760->1785855) req@ffff94a0351f3480 x1875739641802624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.300139:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f3480 with x1875739641802624 ext(1781760->1785855) 00002000:00010000:3.0:1788845045.300145:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f3480 00000100:00100000:3.0:1788845045.300190:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 561 00000100:00100000:3.0:1788845045.300200:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641802624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.300465:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641802624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (709us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.300476:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 561 00002000:00100000:3.0:1788845045.300482:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1781760->1785855) req@ffff94a0351f3480 x1875739641802624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.300490:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f3480 with x1875739641802624 ext(1781760->1785855) 00002000:00010000:3.0:1788845045.300493:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f3480 00000100:00100000:0.0:1788845045.307982:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.308290:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641802752 00000100:00100000:2.0:1788845045.308303:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641802752 00002000:00100000:2.0:1788845045.308365:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1785856->1789951) req@ffff949f03455880 x1875739641802752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.308376:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03455880 with x1875739641802752 ext(1785856->1789951) 00002000:00010000:2.0:1788845045.308382:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03455880 00000100:00100000:2.0:1788845045.308420:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 562 00000100:00100000:2.0:1788845045.308430:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641802752:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.308695:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641802752:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 265us (726us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.308703:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 562 00002000:00100000:2.0:1788845045.308710:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1785856->1789951) req@ffff949f03455880 x1875739641802752/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.308718:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03455880 with x1875739641802752 ext(1785856->1789951) 00002000:00010000:2.0:1788845045.308722:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03455880 00000100:00100000:0.0:1788845045.313134:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.313661:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641802880 00000100:00100000:3.0:1788845045.313673:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641802880 00002000:00100000:2.0:1788845045.313799:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1789952->1794047) req@ffff949f03455f80 x1875739641802880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.313811:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03455f80 with x1875739641802880 ext(1789952->1794047) 00002000:00010000:2.0:1788845045.313817:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03455f80 00000100:00100000:2.0:1788845045.313916:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 563 00000100:00100000:2.0:1788845045.313927:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641802880:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.314381:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641802880:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 454us (1255us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.314389:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 563 00002000:00100000:2.0:1788845045.314395:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1789952->1794047) req@ffff949f03455f80 x1875739641802880/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.314404:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03455f80 with x1875739641802880 ext(1789952->1794047) 00002000:00010000:2.0:1788845045.314407:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03455f80 00000100:00100000:3.0:1788845045.321176:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.321283:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641803008 00000100:00100000:2.0:1788845045.321294:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641803008 00002000:00100000:2.0:1788845045.321352:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1794048->1798143) req@ffff94a035743800 x1875739641803008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.321365:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035743800 with x1875739641803008 ext(1794048->1798143) 00002000:00010000:2.0:1788845045.321370:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035743800 00000100:00100000:2.0:1788845045.321407:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 564 00000100:00100000:2.0:1788845045.321417:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035743800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641803008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.321676:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035743800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641803008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (526us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.321684:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 564 00002000:00100000:2.0:1788845045.321691:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1794048->1798143) req@ffff94a035743800 x1875739641803008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.321699:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035743800 with x1875739641803008 ext(1794048->1798143) 00002000:00010000:2.0:1788845045.321703:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035743800 00000100:00100000:1.0:1788845045.325216:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.325373:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641803136 00000100:00100000:2.0:1788845045.325383:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641803136 00002000:00100000:2.0:1788845045.325428:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1798144->1802239) req@ffff94a0351f1500 x1875739641803136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.325439:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f1500 with x1875739641803136 ext(1798144->1802239) 00002000:00010000:2.0:1788845045.325444:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f1500 00000100:00100000:2.0:1788845045.325472:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 565 00000100:00100000:2.0:1788845045.325481:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641803136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.325719:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641803136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 237us (511us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.325726:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 565 00002000:00100000:2.0:1788845045.325734:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1798144->1802239) req@ffff94a0351f1500 x1875739641803136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.325743:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f1500 with x1875739641803136 ext(1798144->1802239) 00002000:00010000:2.0:1788845045.325747:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f1500 00000100:00100000:0.0:1788845045.329366:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.329536:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641803264 00000100:00100000:2.0:1788845045.329546:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641803264 00002000:00100000:2.0:1788845045.329593:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1802240->1806335) req@ffff949f03455180 x1875739641803264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.329605:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03455180 with x1875739641803264 ext(1802240->1806335) 00002000:00010000:2.0:1788845045.329612:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03455180 00000100:00100000:2.0:1788845045.329654:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 566 00000100:00100000:2.0:1788845045.329664:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641803264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.330116:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641803264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 453us (759us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.330124:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 566 00002000:00100000:2.0:1788845045.330132:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1802240->1806335) req@ffff949f03455180 x1875739641803264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.330142:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03455180 with x1875739641803264 ext(1802240->1806335) 00002000:00010000:2.0:1788845045.330145:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03455180 00010000:00010000:1.0:1788845045.330402:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845045.334132:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.334414:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641803392 00000100:00100000:2.0:1788845045.334424:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641803392 00002000:00100000:2.0:1788845045.334474:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1806336->1810431) req@ffff94a0351f3100 x1875739641803392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.334485:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f3100 with x1875739641803392 ext(1806336->1810431) 00002000:00010000:2.0:1788845045.334491:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f3100 00000100:00100000:2.0:1788845045.334614:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 567 00000100:00100000:2.0:1788845045.334625:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641803392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.334952:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641803392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (760us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.334961:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 567 00002000:00100000:2.0:1788845045.334968:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1806336->1810431) req@ffff94a0351f3100 x1875739641803392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.334977:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0351f3100 with x1875739641803392 ext(1806336->1810431) 00002000:00010000:2.0:1788845045.334981:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0351f3100 00000100:00100000:0.0:1788845045.338051:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.338342:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641803520 00000100:00100000:2.0:1788845045.338352:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641803520 00002000:00100000:2.0:1788845045.338396:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1810432->1814527) req@ffff94a035be5500 x1875739641803520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.338408:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be5500 with x1875739641803520 ext(1810432->1814527) 00002000:00010000:2.0:1788845045.338413:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be5500 00000100:00100000:2.0:1788845045.338486:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 568 00000100:00100000:2.0:1788845045.338497:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641803520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.338734:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641803520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 238us (690us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.338742:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 568 00002000:00100000:2.0:1788845045.338748:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1810432->1814527) req@ffff94a035be5500 x1875739641803520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.338763:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be5500 with x1875739641803520 ext(1810432->1814527) 00002000:00010000:2.0:1788845045.338767:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be5500 00000100:00100000:0.0:1788845045.342459:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.342593:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641803648 00000100:00100000:3.0:1788845045.342605:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641803648 00002000:00100000:3.0:1788845045.342665:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1818623) req@ffff94a035be7100 x1875739641803648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.342677:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be7100 with x1875739641803648 ext(1814528->1818623) 00002000:00010000:3.0:1788845045.342682:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be7100 00000100:00100000:3.0:1788845045.342713:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 569 00000100:00100000:3.0:1788845045.342723:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641803648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.342979:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641803648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (529us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.342986:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 569 00002000:00100000:3.0:1788845045.342993:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1818623) req@ffff94a035be7100 x1875739641803648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.343000:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035be7100 with x1875739641803648 ext(1814528->1818623) 00002000:00010000:3.0:1788845045.343003:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035be7100 00000100:00100000:3.0:1788845045.347088:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.347254:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641803776 00000100:00100000:2.0:1788845045.347264:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641803776 00002000:00100000:2.0:1788845045.347309:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1818624->1822719) req@ffff949f033f7b80 x1875739641803776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.347321:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f7b80 with x1875739641803776 ext(1818624->1822719) 00002000:00010000:2.0:1788845045.347326:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f7b80 00000100:00100000:2.0:1788845045.347357:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 570 00000100:00100000:2.0:1788845045.347366:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641803776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.347613:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641803776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 247us (537us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.347621:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 570 00002000:00100000:2.0:1788845045.347627:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1818624->1822719) req@ffff949f033f7b80 x1875739641803776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.347645:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f7b80 with x1875739641803776 ext(1818624->1822719) 00002000:00010000:2.0:1788845045.347649:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f7b80 00000100:00100000:0.0:1788845045.351650:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.351753:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641803904 00000100:00100000:1.0:1788845045.351763:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641803904 00000100:00100000:1.0:1788845045.351821:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1170 00000100:00100000:1.0:1788845045.351831:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+19:2367:x1875739641803904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.351851:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.351870:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3d1 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.351934:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:1.0:1788845045.351939:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:1.0:1788845045.351950:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 20 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.351969:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 20 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.351983:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757400, rc 0) 00000100:00100000:1.0:1788845045.352044:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2367:x1875739641803904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 214us (413us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845045.352051:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1170 00000100:00100000:1.0:1788845045.355861:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.356029:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641804032 00000100:00100000:2.0:1788845045.356038:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641804032 00000100:00100000:2.0:1788845045.356102:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1012 00000100:00100000:2.0:1788845045.356113:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+22:2368:x1875739641804032:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:2.0:1788845045.356482:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1011804 -> 685309 00000004:00010000:2.0:1788845045.356487:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845045 -> 1788845045 00000004:00010000:2.0:1788845045.356490:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845044 -> 1788845043 00000004:00010000:2.0:1788845045.356492:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845045 -> 1788845045 00000004:00010000:2.0:1788845045.356494:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845045.356627:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2368:x1875739641804032:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 515us (775us total) trans 4294967859 rc 0/0 00000100:00100000:2.0:1788845045.356643:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1012 00000100:00100000:3.0:1788845045.359980:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.360124:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641804160 00000100:00100000:2.0:1788845045.360134:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641804160 00000100:00100000:2.0:1788845045.360274:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1588 00000100:00100000:2.0:1788845045.360285:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641804160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.360305:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.360323:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef51 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3d8 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.360364:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.360401:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660402944:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.360535:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845045.363559:0:5841:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 685309 -> 1011804 00000004:00010000:2.0:1788845045.363568:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845044 00000100:00100000:2.0:1788845045.363593:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660402944:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.363618:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845045.363645:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef51 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d8 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.363665:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.363668:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.363673:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef51 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d8 expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.363689:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773c00, rc 0) 00000100:00100000:2.0:1788845045.363789:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641804160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3506us (3817us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.363798:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1588 00000100:00100000:0.0:1788845045.366872:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.366975:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641804288 00000100:00100000:3.0:1788845045.366985:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641804288 00000100:00100000:3.0:1788845045.367040:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1589 00000100:00100000:3.0:1788845045.367050:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641804288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.367070:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.367090:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef58 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3df expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.367127:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.367177:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660403072:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.367438:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e220c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.370502:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660403072:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845045.370536:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:3.0:1788845045.370555:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef58 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3df expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.370732:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845045.370736:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845045.370742:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef58 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3df expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.370759:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845045.370866:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641804288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3816us (4002us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.370879:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1589 00000100:00100000:1.0:1788845045.374117:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.374277:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641804416 00000100:00100000:2.0:1788845045.374286:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641804416 00000100:00100000:2.0:1788845045.374378:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1590 00000100:00100000:2.0:1788845045.374389:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641804416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.374410:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.374432:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef5f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3e6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.374474:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.374667:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660403200:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.374756:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.377824:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660403200:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.377854:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845045.377874:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef5f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3e6 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.377928:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.377931:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.377937:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef5f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3e6 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.377952:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773c00, rc 0) 00000100:00100000:2.0:1788845045.378018:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641804416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3630us (3909us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.378027:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1590 00000100:00100000:1.0:1788845045.380971:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.381121:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641804544 00000100:00100000:2.0:1788845045.381131:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641804544 00000100:00100000:2.0:1788845045.381196:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1171 00000100:00100000:2.0:1788845045.381206:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641804544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.381225:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.381243:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3ed expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.381286:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 15 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.381296:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 19 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.381303:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:2.0:1788845045.381308:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:2.0:1788845045.381313:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1773568->1777663] (req 1773568->1777663) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 15 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.381354:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 19 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.381375:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 19 pid: 6685 timeout: 534 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.381389:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660403328:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845045.382096:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 15 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.382109:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 15 pid: 7983 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.382121:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660403456:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845045.382167:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.387491:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.387543:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.387680:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641804672 00000100:00100000:0.0:1788845045.387689:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641804672 00010000:00010000:0.0:1788845045.387732:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 20 pid: 6685 timeout: 534 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.387772:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641804800 00000100:00100000:0.0:1788845045.387776:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641804800 00010000:00010000:0.0:1788845045.387790:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 16 pid: 7983 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.387810:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 540 00000100:00100000:0.0:1788845045.387820:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641804672:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845045.387835:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845045.387858:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 20 pid: 6685 timeout: 534 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.387877:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 20 pid: 6685 timeout: 534 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.387922:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845045.387970:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+20:2370:x1875739641804672:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 151us (488us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845045.387977:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 540 00000100:00100000:0.0:1788845045.387994:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 541 00000100:00100000:0.0:1788845045.388001:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641804800:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845045.388009:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845045.388018:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 16 pid: 7983 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.388030:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 16 pid: 7983 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.388043:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:0.0:1788845045.388048:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(11 17) 00010000:00010000:0.0:1788845045.388057:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.388067:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.388109:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 16 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.388124:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660403584:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845045.388184:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.390544:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660403584:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845045.390568:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845045.390623:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641804800:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2622us (3079us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845045.390642:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 541 00010000:00010000:2.0:1788845045.391931:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000163dc567/0xb2187301287aef4a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3d1 expref: 19 pid: 6685 timeout: 534 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.391953:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660403328:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845045.391968:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287aef43 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ca expref: 15 pid: 7983 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.391977:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660403456:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845045.392007:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 14 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.392025:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773200, rc 0) 00000100:00100000:2.0:1788845045.392070:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641804544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10866us (11109us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845045.392078:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1171 00000100:00100000:0.0:1788845045.406203:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.406301:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641804928 00000100:00100000:2.0:1788845045.406310:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641804928 00002000:00100000:2.0:1788845045.406361:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1773568->1777663) req@ffff949f0358e680 x1875739641804928/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.406373:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358e680 with x1875739641804928 ext(1773568->1777663) 00002000:00010000:2.0:1788845045.406379:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358e680 00000100:00100000:2.0:1788845045.406408:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 571 00000100:00100000:2.0:1788845045.406416:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641804928:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.406659:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641804928:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (465us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.406667:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 571 00002000:00100000:2.0:1788845045.406674:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1773568->1777663) req@ffff949f0358e680 x1875739641804928/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.407268:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358e680 with x1875739641804928 ext(1773568->1777663) 00002000:00010000:2.0:1788845045.407274:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358e680 00000100:00100000:1.0:1788845045.419007:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.419522:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641805056 00000100:00100000:2.0:1788845045.419534:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641805056 00002000:00100000:2.0:1788845045.419594:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1777664->1781759) req@ffff949f02364380 x1875739641805056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.419605:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02364380 with x1875739641805056 ext(1777664->1781759) 00002000:00010000:2.0:1788845045.419612:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02364380 00000100:00100000:2.0:1788845045.419660:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 572 00000100:00100000:2.0:1788845045.419672:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641805056:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845045.423611:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641805056:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 3932us (4605us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845045.423625:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 572 00002000:00100000:1.0:1788845045.423645:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1777664->1781759) req@ffff949f02364380 x1875739641805056/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.423662:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02364380 with x1875739641805056 ext(1777664->1781759) 00002000:00010000:1.0:1788845045.423668:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02364380 00000100:00100000:0.0:1788845045.428450:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.428604:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641805184 00000100:00100000:1.0:1788845045.428614:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641805184 00002000:00100000:1.0:1788845045.428678:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1781760->1785855) req@ffff949f0358f480 x1875739641805184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.428757:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358f480 with x1875739641805184 ext(1781760->1785855) 00002000:00010000:1.0:1788845045.428764:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358f480 00000100:00100000:1.0:1788845045.428850:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 573 00000100:00100000:1.0:1788845045.428862:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641805184:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845045.429161:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641805184:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 299us (810us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845045.429168:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 573 00002000:00100000:1.0:1788845045.429176:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1781760->1785855) req@ffff949f0358f480 x1875739641805184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.429183:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358f480 with x1875739641805184 ext(1781760->1785855) 00002000:00010000:1.0:1788845045.429186:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358f480 00000100:00100000:0.0:1788845045.432942:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.434030:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641805312 00000100:00100000:1.0:1788845045.434042:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641805312 00002000:00100000:1.0:1788845045.434094:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1785856->1789951) req@ffff949f0358f800 x1875739641805312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.434105:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358f800 with x1875739641805312 ext(1785856->1789951) 00002000:00010000:1.0:1788845045.434111:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358f800 00000100:00100000:1.0:1788845045.434192:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 574 00000100:00100000:1.0:1788845045.434202:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641805312:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845045.434603:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641805312:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 401us (1669us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845045.434612:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 574 00002000:00100000:1.0:1788845045.434619:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1785856->1789951) req@ffff949f0358f800 x1875739641805312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.434627:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0358f800 with x1875739641805312 ext(1785856->1789951) 00002000:00010000:1.0:1788845045.434642:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0358f800 00000100:00100000:1.0:1788845045.439044:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.439127:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641805440 00000100:00100000:0.0:1788845045.439138:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641805440 00002000:00100000:0.0:1788845045.439195:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1789952->1794047) req@ffff949f02367100 x1875739641805440/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.439207:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02367100 with x1875739641805440 ext(1789952->1794047) 00002000:00010000:0.0:1788845045.439212:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02367100 00000100:00100000:0.0:1788845045.439240:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 575 00000100:00100000:0.0:1788845045.439248:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641805440:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845045.439672:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641805440:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 422us (636us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845045.439688:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 575 00002000:00100000:0.0:1788845045.439694:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1789952->1794047) req@ffff949f02367100 x1875739641805440/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.439702:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02367100 with x1875739641805440 ext(1789952->1794047) 00002000:00010000:0.0:1788845045.439706:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02367100 00000100:00100000:1.0:1788845045.450519:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.450733:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641805568 00000100:00100000:2.0:1788845045.450743:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641805568 00002000:00100000:2.0:1788845045.450798:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1794048->1798143) req@ffff94a035929500 x1875739641805568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.450810:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035929500 with x1875739641805568 ext(1794048->1798143) 00002000:00010000:2.0:1788845045.450816:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035929500 00000100:00100000:2.0:1788845045.450849:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 576 00000100:00100000:2.0:1788845045.450859:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641805568:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.451096:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641805568:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 238us (598us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.451104:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 576 00002000:00100000:2.0:1788845045.451111:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1794048->1798143) req@ffff94a035929500 x1875739641805568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.451119:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035929500 with x1875739641805568 ext(1794048->1798143) 00002000:00010000:2.0:1788845045.451122:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035929500 00000100:00100000:1.0:1788845045.457217:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.457316:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641805696 00000100:00100000:3.0:1788845045.457326:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641805696 00002000:00100000:3.0:1788845045.457379:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1798144->1802239) req@ffff94a035929180 x1875739641805696/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.457390:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035929180 with x1875739641805696 ext(1798144->1802239) 00002000:00010000:3.0:1788845045.457396:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035929180 00000100:00100000:3.0:1788845045.457433:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 577 00000100:00100000:3.0:1788845045.457443:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641805696:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.457709:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641805696:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (501us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.457717:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 577 00002000:00100000:3.0:1788845045.457724:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1798144->1802239) req@ffff94a035929180 x1875739641805696/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.457732:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035929180 with x1875739641805696 ext(1798144->1802239) 00002000:00010000:3.0:1788845045.457735:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035929180 00000100:00100000:1.0:1788845045.466466:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.466656:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641805824 00000100:00100000:2.0:1788845045.466668:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641805824 00002000:00100000:2.0:1788845045.466722:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1802240->1806335) req@ffff94a03592b100 x1875739641805824/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.466735:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03592b100 with x1875739641805824 ext(1802240->1806335) 00002000:00010000:2.0:1788845045.466742:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03592b100 00000100:00100000:2.0:1788845045.466774:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 578 00000100:00100000:2.0:1788845045.466783:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641805824:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.467053:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641805824:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 270us (595us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.467062:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 578 00002000:00100000:2.0:1788845045.467068:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1802240->1806335) req@ffff94a03592b100 x1875739641805824/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.467076:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03592b100 with x1875739641805824 ext(1802240->1806335) 00002000:00010000:2.0:1788845045.467080:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03592b100 00000100:00100000:1.0:1788845045.472084:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.472184:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641805952 00000100:00100000:3.0:1788845045.472194:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641805952 00002000:00100000:3.0:1788845045.472243:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1806336->1810431) req@ffff949f01fc7480 x1875739641805952/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.472254:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc7480 with x1875739641805952 ext(1806336->1810431) 00002000:00010000:3.0:1788845045.472261:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc7480 00000100:00100000:3.0:1788845045.472298:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 579 00000100:00100000:3.0:1788845045.472308:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641805952:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.472682:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641805952:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 375us (607us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.472691:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 579 00002000:00100000:3.0:1788845045.472698:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1806336->1810431) req@ffff949f01fc7480 x1875739641805952/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.472705:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc7480 with x1875739641805952 ext(1806336->1810431) 00002000:00010000:3.0:1788845045.472709:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc7480 00000100:00100000:1.0:1788845045.476787:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.477065:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641806080 00000100:00100000:2.0:1788845045.477076:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641806080 00002000:00100000:2.0:1788845045.477127:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1810432->1814527) req@ffff949f01fc5180 x1875739641806080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.477139:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc5180 with x1875739641806080 ext(1810432->1814527) 00002000:00010000:2.0:1788845045.477145:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc5180 00000100:00100000:2.0:1788845045.477299:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 580 00000100:00100000:2.0:1788845045.477310:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641806080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845045.479726:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641806080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 2406us (2939us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845045.479738:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 580 00002000:00100000:2.0:1788845045.479750:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1810432->1814527) req@ffff949f01fc5180 x1875739641806080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845045.479759:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01fc5180 with x1875739641806080 ext(1810432->1814527) 00002000:00010000:2.0:1788845045.479765:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01fc5180 00000100:00100000:1.0:1788845045.489280:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.489701:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641806208 00000100:00100000:0.0:1788845045.489712:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641806208 00002000:00100000:0.0:1788845045.489765:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1818623) req@ffff94a00891b800 x1875739641806208/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.489776:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a00891b800 with x1875739641806208 ext(1814528->1818623) 00002000:00010000:0.0:1788845045.489782:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a00891b800 00000100:00100000:0.0:1788845045.489815:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 581 00000100:00100000:0.0:1788845045.489825:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641806208:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845045.490169:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641806208:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 344us (901us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845045.490178:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 581 00002000:00100000:0.0:1788845045.490185:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1814528->1818623) req@ffff94a00891b800 x1875739641806208/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.490193:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a00891b800 with x1875739641806208 ext(1814528->1818623) 00002000:00010000:0.0:1788845045.490196:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a00891b800 00000100:00100000:1.0:1788845045.502764:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.502857:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641806336 00000100:00100000:3.0:1788845045.502867:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641806336 00002000:00100000:3.0:1788845045.502959:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1818624->1822719) req@ffff949f0383d180 x1875739641806336/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.502970:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383d180 with x1875739641806336 ext(1818624->1822719) 00002000:00010000:3.0:1788845045.502976:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383d180 00000100:00100000:3.0:1788845045.503011:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 582 00000100:00100000:3.0:1788845045.503020:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641806336:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.503296:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641806336:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 274us (540us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.503303:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 582 00002000:00100000:3.0:1788845045.503310:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1818624->1822719) req@ffff949f0383d180 x1875739641806336/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.503319:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383d180 with x1875739641806336 ext(1818624->1822719) 00002000:00010000:3.0:1788845045.503322:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383d180 00000100:00100000:1.0:1788845045.508116:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.508442:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641806464 00000100:00100000:0.0:1788845045.508454:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641806464 00002000:00100000:0.0:1788845045.508517:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1822720->1826815) req@ffff949f0383f100 x1875739641806464/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.508528:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383f100 with x1875739641806464 ext(1822720->1826815) 00002000:00010000:0.0:1788845045.508535:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383f100 00000100:00100000:0.0:1788845045.508614:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 583 00000100:00100000:0.0:1788845045.508624:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641806464:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845045.508926:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641806464:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 303us (819us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845045.508934:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 583 00002000:00100000:0.0:1788845045.508941:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1822720->1826815) req@ffff949f0383f100 x1875739641806464/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.508948:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0383f100 with x1875739641806464 ext(1822720->1826815) 00002000:00010000:0.0:1788845045.508951:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0383f100 00000100:00100000:1.0:1788845045.513354:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.513493:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641806592 00000100:00100000:3.0:1788845045.513517:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641806592 00002000:00100000:3.0:1788845045.513571:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1826816->1830911) req@ffff94a0360ef100 x1875739641806592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.513582:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ef100 with x1875739641806592 ext(1826816->1830911) 00002000:00010000:3.0:1788845045.513587:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ef100 00000100:00100000:3.0:1788845045.513668:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 584 00000100:00100000:3.0:1788845045.513679:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641806592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.513931:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641806592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (1049us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.513939:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 584 00002000:00100000:3.0:1788845045.513947:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1826816->1830911) req@ffff94a0360ef100 x1875739641806592/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.513954:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ef100 with x1875739641806592 ext(1826816->1830911) 00002000:00010000:3.0:1788845045.513958:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ef100 00000100:00100000:1.0:1788845045.528397:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.528479:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641806720 00000100:00100000:3.0:1788845045.528490:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641806720 00002000:00100000:3.0:1788845045.528556:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1830912->1835007) req@ffff94a0360ec380 x1875739641806720/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.528568:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ec380 with x1875739641806720 ext(1830912->1835007) 00002000:00010000:3.0:1788845045.528574:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ec380 00000100:00100000:3.0:1788845045.528607:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 585 00000100:00100000:3.0:1788845045.528616:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641806720:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.528857:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641806720:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 240us (468us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.528868:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 585 00002000:00100000:3.0:1788845045.528875:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1830912->1835007) req@ffff94a0360ec380 x1875739641806720/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.528924:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a0360ec380 with x1875739641806720 ext(1830912->1835007) 00002000:00010000:3.0:1788845045.528928:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a0360ec380 00000100:00100000:1.0:1788845045.535452:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.535555:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641806848 00000100:00100000:0.0:1788845045.535566:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641806848 00002000:00100000:0.0:1788845045.535615:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1835008->1839103) req@ffff949f01dbb800 x1875739641806848/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.535638:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01dbb800 with x1875739641806848 ext(1835008->1839103) 00002000:00010000:0.0:1788845045.535644:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01dbb800 00000100:00100000:0.0:1788845045.535680:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 586 00000100:00100000:0.0:1788845045.535690:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641806848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845045.536038:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641806848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 347us (593us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845045.536046:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 586 00002000:00100000:0.0:1788845045.536053:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1835008->1839103) req@ffff949f01dbb800 x1875739641806848/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845045.536062:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01dbb800 with x1875739641806848 ext(1835008->1839103) 00002000:00010000:0.0:1788845045.536066:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01dbb800 00000100:00100000:1.0:1788845045.550257:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.550348:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641806976 00000100:00100000:3.0:1788845045.550373:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641806976 00002000:00100000:3.0:1788845045.550424:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1773568->1839103) req@ffff949f01db8a80 x1875739641806976/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.550436:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db8a80 with x1875739641806976 ext(1773568->1839103) 00002000:00010000:3.0:1788845045.550442:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db8a80 00000100:00100000:3.0:1788845045.550475:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 587 00000100:00100000:3.0:1788845045.550485:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641806976:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845045.558560:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641806976:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 8064us (8302us total) trans 4294967507 rc 0/0 00000100:00100000:3.0:1788845045.558575:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 587 00002000:00100000:3.0:1788845045.558589:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1773568->1839103) req@ffff949f01db8a80 x1875739641806976/t4294967507(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:151/0 lens 488/448 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.558601:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01db8a80 with x1875739641806976 ext(1773568->1839103) 00002000:00010000:3.0:1788845045.558607:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01db8a80 00000100:00100000:0.0:1788845045.561619:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.567582:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641807104 00000100:00100000:1.0:1788845045.567602:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641807104 00000100:00100000:1.0:1788845045.567744:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1172 00000100:00100000:1.0:1788845045.567755:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641807104:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:2.0:1788845045.658717:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967493 is committed 00000001:00080000:2.0:1788845045.658750:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967506 is committed 00000001:00080000:2.0:1788845045.658762:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967502 is committed 00000001:00080000:2.0:1788845045.658772:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967505 is committed 00000001:00080000:2.0:1788845045.662818:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967507 is committed 00000100:00100000:0.0:1788845045.665452:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641807104:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 97686us (103829us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845045.665471:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1172 00000100:00100000:1.0:1788845045.669987:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.670295:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641807232 00000100:00100000:2.0:1788845045.670305:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641807232 00000100:00100000:2.0:1788845045.670356:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1591 00000100:00100000:2.0:1788845045.670366:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641807232:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.670389:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.670410:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef6d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3f4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.670455:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.670492:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660403712:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.670585:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.681357:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660403712:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.681386:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845045.681405:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef6d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3f4 expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.681432:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.681435:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.681440:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef6d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3f4 expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.681467:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773c00, rc 0) 00000100:00100000:2.0:1788845045.681649:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01dba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641807232:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11283us (11673us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.681659:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1591 00000100:00100000:0.0:1788845045.687713:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.687910:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641807360 00000100:00100000:3.0:1788845045.687920:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641807360 00000100:00100000:3.0:1788845045.687974:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1592 00000100:00100000:3.0:1788845045.687984:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641807360:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.688008:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.688027:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef74 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa3fb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.688062:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.688100:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660403840:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.688188:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.692607:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660403840:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845045.692645:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:3.0:1788845045.692667:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef74 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3fb expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.692704:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845045.692707:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845045.692712:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef74 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3fb expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.692730:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:3.0:1788845045.692803:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641807360:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4819us (5101us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.692812:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1592 00000100:00100000:1.0:1788845045.704487:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.704706:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641807488 00000100:00100000:3.0:1788845045.704717:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641807488 00000100:00100000:3.0:1788845045.704784:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1593 00000100:00100000:3.0:1788845045.704794:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641807488:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.704817:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.704837:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef7b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa402 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.704873:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.704938:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660403968:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.705008:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e226c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.712454:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660403968:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.712486:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845045.712507:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef7b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa402 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.712529:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.712532:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.712537:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030800/0xb2187301287aef7b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa402 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.712554:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030800, rc 0) 00000100:00100000:2.0:1788845045.712660:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+25:2368:x1875739641807488:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7866us (8180us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.712670:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1593 00000100:00100000:0.0:1788845045.717005:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.717123:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641807616 00000100:00100000:2.0:1788845045.717135:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641807616 00000100:00100000:2.0:1788845045.717194:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1594 00000100:00100000:2.0:1788845045.717203:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641807616:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.717223:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.717242:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef82 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa409 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.717280:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.717321:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660404096:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.717403:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.729185:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660404096:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.729214:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845045.729235:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef82 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa409 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.729257:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.729260:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.729267:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aef82 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa409 expref: 26 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.729282:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773c00, rc 0) 00000100:00100000:2.0:1788845045.729339:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010483480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+25:2367:x1875739641807616:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12136us (12342us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.729349:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1594 00000100:00100000:1.0:1788845045.735225:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.735449:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641807744 00000100:00100000:3.0:1788845045.735459:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641807744 00000100:00100000:2.0:1788845045.735538:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1173 00000100:00100000:2.0:1788845045.735551:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641807744:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.735571:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.735596:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cefb4552/0xb2187301287aef89 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa410 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.735666:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660404224:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.736044:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00002000:00010000:2.0:1788845045.737979:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845044 -> 1788845045 00002000:00010000:2.0:1788845045.737990:0:6684:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 57 -> 269 00000100:00100000:2.0:1788845045.738013:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660404224:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845045.738045:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cefb4552/0xb2187301287aef89 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa410 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.738067:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cefb4552/0xb2187301287aef89 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa410 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.738080:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773c00, rc 0) 00000100:00100000:2.0:1788845045.738135:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641807744:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2587us (2924us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.738142:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1173 00000100:00100000:1.0:1788845045.745200:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.745303:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641807872 00000100:00100000:0.0:1788845045.745317:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641807872 00000100:00100000:0.0:1788845045.745388:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1174 00000100:00100000:0.0:1788845045.745399:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641807872:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845045.745420:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845045.745438:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287aef90 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa417 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.745511:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010481c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660404352:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845045.745600:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.754240:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010481c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660404352:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845045.754293:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287aef90 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa417 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.754319:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287aef90 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa417 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.754332:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006800, rc 0) 00000100:00100000:0.0:1788845045.754438:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641807872:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9040us (9246us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845045.754447:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1174 00000100:00100000:2.0:1788845045.759319:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.759571:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641808000 00000100:00100000:3.0:1788845045.759581:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641808000 00000100:00100000:3.0:1788845045.759647:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1595 00000100:00100000:3.0:1788845045.759657:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+25:14891:x1875739641808000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.759745:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.759769:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa41e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.759804:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.759814:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.759854:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.759877:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 22 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.760033:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660404480:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845045.760287:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.764749:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.764847:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660404480:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845045.764908:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.764936:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.764940:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.764945:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.764957:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030000, rc 0) 00000100:00100000:2.0:1788845045.765020:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:14891:x1875739641808000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5363us (5708us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845045.765029:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1595 00000100:00100000:1.0:1788845045.765266:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641808128 00000100:00100000:1.0:1788845045.765274:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641808128 00002000:00100000:1.0:1788845045.765323:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010481c00 x1875739641808128/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 504/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.765333:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010481c00 with x1875739641808128 00010000:00010000:1.0:1788845045.765341:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 23 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.765531:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 23 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845045.765540:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a010481c00 00000100:00100000:1.0:1788845045.765690:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 38 00000100:00100000:1.0:1788845045.765702:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+23:2369:x1875739641808128:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845045.782847:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1011804 -> 1011804 00000004:00010000:1.0:1788845045.782856:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845045 -> 1788845045 00000004:00010000:1.0:1788845045.782859:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845044 -> 1788845043 00000004:00010000:1.0:1788845045.782861:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845045 -> 1788845045 00000004:00010000:1.0:1788845045.782864:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:1.0:1788845045.782963:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2369:x1875739641808128:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 17260us (18224us total) trans 4294967860 rc 0/0 00000100:00100000:1.0:1788845045.782973:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 38 00002000:00100000:1.0:1788845045.782985:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010481c00 x1875739641808128/t4294967860(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 504/448 e 0 to 0 dl 1788845056 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845045.782996:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010481c00 with x1875739641808128 00010000:00010000:1.0:1788845045.783006:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 25 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.783026:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 25 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845045.783034:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff94a010481c00 00000100:00100000:0.0:1788845045.788189:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.788260:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641808256 00000100:00100000:3.0:1788845045.788270:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641808256 00010000:00010000:3.0:1788845045.788311:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 25 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.788339:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 542 00000100:00100000:3.0:1788845045.788349:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641808256:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845045.788362:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845045.788388:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 25 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.788406:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 25 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.788428:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031800/0xb2187301287aee39 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ce expref: 25 pid: 5841 timeout: 534 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.788445:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845045.788451:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.788467:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.788475:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.788527:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.788541:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660404608:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845045.789055:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.798909:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.799004:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641808384 00000100:00100000:0.0:1788845045.799015:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641808384 00000100:00100000:0.0:1788845045.799087:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1013 00000100:00100000:0.0:1788845045.799097:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641808384:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845045.799367:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1011804 -> 1011804 00000004:00010000:0.0:1788845045.799372:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845045 -> 1788845045 00000004:00010000:0.0:1788845045.799375:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:0.0:1788845045.799377:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845045 -> 1788845045 00000004:00010000:0.0:1788845045.799379:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845045.799465:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641808384:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 369us (589us total) trans 4294967861 rc 0/0 00000100:00100000:0.0:1788845045.799473:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1013 00000100:00100000:3.0:1788845045.806550:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660404608:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845045.806579:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845045.806948:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.806987:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.807059:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641808640 00000100:00100000:3.0:1788845045.807066:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641808640 00000100:00100000:3.0:1788845045.807125:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1175 00000100:00100000:3.0:1788845045.807135:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641808640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.807153:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.807169:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287aef9e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa42c expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.807222:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660404736:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.807311:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22840 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.807913:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641808512 00000100:00100000:1.0:1788845045.807918:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641808512 00000100:00100000:1.0:1788845045.807959:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1596 00000100:00100000:1.0:1788845045.807967:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641808512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.807984:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.808000:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287aefa5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa425 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.808033:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.808060:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660404864:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.808112:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.808858:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:2368:x1875739641808256:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 20505us (20673us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845045.808870:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 542 00000100:00100000:3.0:1788845045.825766:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660404736:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845045.825813:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287aef9e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa42c expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.825841:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287aef9e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa42c expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.825855:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:2.0:1788845045.827541:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641808640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 20392us (20538us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.827554:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1175 00000004:00010000:1.0:1788845045.834871:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845045 00000100:00100000:1.0:1788845045.834943:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660404864:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845045.834972:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:1.0:1788845045.834994:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287aefa5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa425 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.835017:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845045.835020:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845045.835026:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757c00/0xb2187301287aefa5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa425 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.835043:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757c00, rc 0) 00000100:00100000:1.0:1788845045.835114:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2367:x1875739641808512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 27146us (28166us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845045.835123:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1596 00000100:00100000:0.0:1788845045.839328:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.839458:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641808768 00000100:00100000:1.0:1788845045.839468:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641808768 00000100:00100000:1.0:1788845045.839542:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1597 00000100:00100000:1.0:1788845045.839552:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2367:x1875739641808768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.839576:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.839598:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aefac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa433 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.839648:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.839702:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660404992:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.839796:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562c840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.842559:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.842695:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641808896 00000100:00100000:1.0:1788845045.842705:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641808896 00000100:00100000:1.0:1788845045.842758:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1176 00000100:00100000:1.0:1788845045.842933:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641808896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845045.842952:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845045.842968:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009eed212f/0xb2187301287aefb3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa43a expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845045.843031:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03beab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660405120:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845045.843110:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.859530:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660404992:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845045.859561:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:1.0:1788845045.859581:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aefac lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa433 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.859605:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845045.859608:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845045.859614:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757800/0xb2187301287aefac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa433 expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845045.859639:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757800, rc 0) 00000100:00100000:1.0:1788845045.859710:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2367:x1875739641808768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 20158us (20393us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845045.859719:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1597 00000100:00100000:2.0:1788845045.864752:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03beab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660405120:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845045.864808:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009eed212f/0xb2187301287aefb3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa43a expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.864833:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009eed212f/0xb2187301287aefb3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa43a expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.864847:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756800, rc 0) 00000100:00100000:2.0:1788845045.864940:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641808896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 22171us (22389us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.864949:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1176 00000100:00100000:0.0:1788845045.882613:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845045.887322:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641809024 00000100:00100000:1.0:1788845045.887339:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641809024 00000100:00100000:1.0:1788845045.887493:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 588 00000100:00100000:1.0:1788845045.887514:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641809024:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:1.0:1788845045.887863:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1788845045.887868:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845045 -> 1788845045 00002000:00010000:1.0:1788845045.887870:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845045 -> 1788845044 00002000:00010000:1.0:1788845045.887873:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845045 -> 1788845045 00002000:00010000:1.0:1788845045.887875:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 269 -> 269 00000100:00100000:1.0:1788845045.887975:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641809024:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 462us (5833us total) trans 4294967508 rc 0/0 00000100:00100000:1.0:1788845045.887984:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 588 00000100:00100000:0.0:1788845045.895282:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.895321:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641809152 00000100:00100000:0.0:1788845045.895331:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641809152 00000100:00100000:0.0:1788845045.895386:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1598 00000100:00100000:0.0:1788845045.895396:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2368:x1875739641809152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845045.895418:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845045.895438:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287aefba lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa441 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.895478:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.895530:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660405248:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845045.895610:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18840 going to sleep for 11 seconds 00000100:00100000:0.0:1788845045.912251:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.912490:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660405248:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845045.912523:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:3.0:1788845045.912541:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287aefba lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa441 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.912562:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845045.912565:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845045.912570:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006400/0xb2187301287aefba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa441 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.912587:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:3.0:1788845045.912658:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2368:x1875739641809152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17261us (17386us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.912666:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1598 00000100:00100000:2.0:1788845045.921806:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641809280 00000100:00100000:2.0:1788845045.921821:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641809280 00000100:00100000:2.0:1788845045.922155:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1177 00000100:00100000:2.0:1788845045.922166:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641809280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.922186:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.922206:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000034594ef8/0xb2187301287aefc1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa448 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.922270:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660405376:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.922365:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00002000:00010000:3.0:1788845045.929650:0:6684:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845044 -> 1788845045 00000100:00100000:3.0:1788845045.929692:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03447800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660405376:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845045.929728:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000034594ef8/0xb2187301287aefc1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa448 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.929752:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034594ef8/0xb2187301287aefc1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa448 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.929865:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773400, rc 0) 00000100:00100000:3.0:1788845045.929938:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641809280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7774us (17695us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.929947:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1177 00000100:00100000:0.0:1788845045.933705:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.935162:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641809408 00000100:00100000:2.0:1788845045.935175:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641809408 00000100:00100000:2.0:1788845045.935236:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1599 00000100:00100000:2.0:1788845045.935246:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641809408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.935274:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.935295:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aefc8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa44f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.935329:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.935368:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660405504:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845045.935457:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.936783:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.937172:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641809536 00000100:00100000:3.0:1788845045.937180:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641809536 00000100:00100000:3.0:1788845045.937230:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1178 00000100:00100000:3.0:1788845045.937238:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641809536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845045.937262:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845045.937279:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287aefcf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa456 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845045.937327:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660405632:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845045.937399:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845045.941526:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660405632:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845045.941570:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287aefcf lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa456 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.941596:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287aefcf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa456 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845045.941616:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:2.0:1788845045.942269:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660405504:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845045.942290:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845045.942305:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aefc8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa44f expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.942322:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845045.942325:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845045.942330:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773c00/0xb2187301287aefc8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa44f expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.942343:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773c00, rc 0) 00000100:00100000:2.0:1788845045.942383:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641809408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7138us (8687us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845045.942392:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1599 00000100:00100000:3.0:1788845045.944094:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641809536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4442us (4901us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845045.944112:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1178 00000100:00100000:3.0:1788845045.954412:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845045.954634:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641809664 00000100:00100000:2.0:1788845045.954647:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641809664 00000100:00100000:2.0:1788845045.954710:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1179 00000100:00100000:2.0:1788845045.954721:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641809664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845045.954740:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845045.954758:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa45d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.954789:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.954798:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(11 17) 00010000:00010000:2.0:1788845045.954802:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:2.0:1788845045.954807:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1216512->1236991] (req 1216512->1236991) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.954844:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 6 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.954864:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 6 pid: 6684 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845045.954880:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660405760:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845045.954981:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845045.963842:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660405760:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845045.963906:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1216512->1236991] (req 1216512->1236991) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845045.963930:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773000, rc 0) 00000100:00100000:2.0:1788845045.963986:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641809664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9267us (9584us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845045.963995:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1179 00000100:00100000:1.0:1788845045.967484:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845045.967635:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641809792 00000100:00100000:0.0:1788845045.967644:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641809792 00010000:00010000:0.0:1788845045.967691:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 7 pid: 6684 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.967734:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 543 00000100:00100000:0.0:1788845045.967743:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641809792:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845045.967757:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845045.967986:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 7 pid: 6684 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.968006:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 7 pid: 6684 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.968032:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b3460c93/0xb2187301287aef66 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa3ed expref: 7 pid: 6684 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.968048:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:0.0:1788845045.968053:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:0.0:1788845045.968061:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.968071:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845045.968108:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845045.968122:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f827480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660405888:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845045.968249:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c189c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845045.972229:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02483800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660406144:0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.972321:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845045.972358:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660406656:0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.972385:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845045.972419:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788845045.972425:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660406272:0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.972447:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845045.972562:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660406400:0@lo:400:kworker.0 00000100:00100000:0.0:1788845045.972593:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845045.972614:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1788845045.972631:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660406016:0@lo:400:kworker.0 00000100:00100000:0.0:1788845045.972660:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845045.972683:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660406528:0@lo:400:kworker.0 00000100:00100000:0.0:1788845045.972704:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845045.972730:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660406144 00000100:00100000:0.0:1788845045.972737:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660406144 00000100:00100000:0.0:1788845045.972787:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660406272 00000100:00100000:0.0:1788845045.972791:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660406272 00000100:00100000:0.0:1788845045.972813:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660406528 00000100:00100000:0.0:1788845045.972815:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660406528 00000100:00100000:0.0:1788845045.972831:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:0.0:1788845045.972839:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660406144:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845045.972916:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660406144:12345-0@lo:400:kworker.0 Request processed in 77us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845045.972924:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:0.0:1788845045.972942:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:0.0:1788845045.972948:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660406272:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845045.972979:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660406272:12345-0@lo:400:kworker.0 Request processed in 30us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845045.972984:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:0.0:1788845045.972996:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:0.0:1788845045.973001:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660406528:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845045.973031:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660406528:12345-0@lo:400:kworker.0 Request processed in 30us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845045.973037:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00080000:0.0:1788845045.973052:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845045.973081:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660406528:0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.973586:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660406656 00000100:00100000:1.0:1788845045.973591:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660406656 00000100:00100000:1.0:1788845045.973643:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660406400 00000100:00100000:1.0:1788845045.973647:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660406400 00000100:00100000:1.0:1788845045.973670:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:1.0:1788845045.973676:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660406656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.973713:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660406656:12345-0@lo:400:kworker.0 Request processed in 37us (1327us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845045.973719:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:1.0:1788845045.973729:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:1.0:1788845045.973734:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660406400:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.973760:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660406400:12345-0@lo:400:kworker.0 Request processed in 26us (1167us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845045.973764:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:1.0:1788845045.973779:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660406016 00000100:00100000:1.0:1788845045.973782:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660406016 00000100:00100000:1.0:1788845045.973805:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:1.0:1788845045.973810:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660406016:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.973836:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660406016:12345-0@lo:400:kworker.0 Request processed in 25us (1176us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845045.973842:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:1.0:1788845045.973909:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660406272:0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.973923:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02483800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660406144:0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.973933:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660406656:0@lo:400:kworker.0 00000100:00080000:0.0:1788845045.974502:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1788845045.974507:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845045.974531:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660406016:0@lo:400:kworker.0 00000100:00100000:0.0:1788845045.974544:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660406400:0@lo:400:kworker.0 00000100:00100000:1.0:1788845045.984445:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.984534:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641809920 00000100:00100000:3.0:1788845045.984544:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641809920 00002000:00100000:3.0:1788845045.984596:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1220607) req@ffff949f01c4b800 x1875739641809920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.984607:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c4b800 with x1875739641809920 ext(1216512->1220607) 00002000:00010000:3.0:1788845045.984614:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c4b800 00000100:00100000:3.0:1788845045.984658:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 589 00000100:00100000:3.0:1788845045.985060:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641809920:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.985362:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641809920:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 693us (927us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.985370:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 589 00002000:00100000:3.0:1788845045.985378:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1220607) req@ffff949f01c4b800 x1875739641809920/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.985385:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c4b800 with x1875739641809920 ext(1216512->1220607) 00002000:00010000:3.0:1788845045.985389:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c4b800 00000100:00100000:0.0:1788845045.985682:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f827480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660405888:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845045.985708:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845045.985923:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641809792:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 18180us (18448us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845045.985933:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 543 00000100:00100000:1.0:1788845045.989689:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.989961:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641810048 00000100:00100000:3.0:1788845045.989972:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641810048 00002000:00100000:3.0:1788845045.990021:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1220608->1224703) req@ffff949f01c48380 x1875739641810048/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.990033:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c48380 with x1875739641810048 ext(1220608->1224703) 00002000:00010000:3.0:1788845045.990038:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c48380 00000100:00100000:3.0:1788845045.990069:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 590 00000100:00100000:3.0:1788845045.990079:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641810048:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.990332:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641810048:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 252us (660us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.990340:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 590 00002000:00100000:3.0:1788845045.990347:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1220608->1224703) req@ffff949f01c48380 x1875739641810048/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.990355:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c48380 with x1875739641810048 ext(1220608->1224703) 00002000:00010000:3.0:1788845045.990359:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c48380 00000100:00100000:1.0:1788845045.994334:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.994465:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641810176 00000100:00100000:3.0:1788845045.994476:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641810176 00002000:00100000:3.0:1788845045.994592:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1224704->1228799) req@ffff949f01c48000 x1875739641810176/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.994603:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c48000 with x1875739641810176 ext(1224704->1228799) 00002000:00010000:3.0:1788845045.994609:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c48000 00000100:00100000:3.0:1788845045.994715:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 591 00000100:00100000:3.0:1788845045.994728:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641810176:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.995220:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641810176:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 493us (894us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.995228:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 591 00002000:00100000:3.0:1788845045.995236:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1224704->1228799) req@ffff949f01c48000 x1875739641810176/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.995245:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01c48000 with x1875739641810176 ext(1224704->1228799) 00002000:00010000:3.0:1788845045.995248:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01c48000 00000100:00100000:0.0:1788845045.998921:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845045.999011:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641810304 00000100:00100000:3.0:1788845045.999022:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641810304 00002000:00100000:3.0:1788845045.999070:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1228800->1232895) req@ffff94a03f827480 x1875739641810304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/0 e 0 to 0 dl 1788845056 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845045.999081:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f827480 with x1875739641810304 ext(1228800->1232895) 00002000:00010000:3.0:1788845045.999086:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f827480 00000100:00100000:3.0:1788845045.999228:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 592 00000100:00100000:3.0:1788845045.999239:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641810304:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845045.999475:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641810304:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 237us (562us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845045.999483:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 592 00002000:00100000:3.0:1788845045.999490:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1228800->1232895) req@ffff94a03f827480 x1875739641810304/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:151/0 lens 488/4536 e 0 to 0 dl 1788845056 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.002300:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f827480 with x1875739641810304 ext(1228800->1232895) 00002000:00010000:3.0:1788845046.002310:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03f827480 00000100:00100000:1.0:1788845046.005229:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.005339:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641810432 00000100:00100000:0.0:1788845046.005350:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641810432 00002000:00100000:0.0:1788845046.005402:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1232896->1236991) req@ffff949f02483800 x1875739641810432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.005413:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02483800 with x1875739641810432 ext(1232896->1236991) 00002000:00010000:0.0:1788845046.005419:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02483800 00000100:00100000:0.0:1788845046.007650:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 593 00000100:00100000:0.0:1788845046.007682:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641810432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.007981:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02483800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641810432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 316us (2758us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.007989:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 593 00002000:00100000:0.0:1788845046.007999:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1232896->1236991) req@ffff949f02483800 x1875739641810432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.008009:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02483800 with x1875739641810432 ext(1232896->1236991) 00002000:00010000:0.0:1788845046.008017:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02483800 00000100:00100000:1.0:1788845046.012569:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.013074:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641810560 00000100:00100000:2.0:1788845046.013084:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641810560 00000100:00100000:2.0:1788845046.013134:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1180 00000100:00100000:2.0:1788845046.013154:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641810560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.013171:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.013189:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa464 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.013218:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:2.0:1788845046.013222:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:2.0:1788845046.013232:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.013248:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.013263:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772c00, rc 0) 00000100:00100000:2.0:1788845046.013324:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641810560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 172us (766us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845046.013331:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1180 00000100:00100000:3.0:1788845046.019183:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.019290:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641810688 00000100:00100000:3.0:1788845046.019299:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641810688 00000100:00100000:3.0:1788845046.019347:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1600 00000100:00100000:3.0:1788845046.019357:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641810688:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845046.019381:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845046.019402:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aefe4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa46b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.019440:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.019480:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660406784:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845046.019551:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22000 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.024123:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660406784:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845046.024162:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:0.0:1788845046.024184:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aefe4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa46b expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.024204:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.024207:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845046.024213:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030600/0xb2187301287aefe4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa46b expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.024235:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02030600, rc 0) 00000100:00100000:0.0:1788845046.024301:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641810688:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4945us (5148us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845046.024310:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1600 00000100:00100000:2.0:1788845046.028234:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.029067:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641810816 00000100:00100000:0.0:1788845046.029077:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641810816 00000100:00100000:0.0:1788845046.029149:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1601 00000100:00100000:0.0:1788845046.029161:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641810816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.029183:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.029203:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aefeb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa472 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.029243:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.029302:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f825500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660406912:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845046.029381:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.045050:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f825500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660406912:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845046.045077:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:0.0:1788845046.045098:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aefeb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa472 expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.045120:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.045123:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845046.045129:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287aefeb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa472 expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.045160:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845046.045223:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641810816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16064us (17000us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845046.045233:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1601 00000100:00100000:1.0:1788845046.048430:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.048521:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641810944 00000100:00100000:0.0:1788845046.048531:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641810944 00000100:00100000:0.0:1788845046.048595:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1181 00000100:00100000:0.0:1788845046.048604:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641810944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.048630:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.048649:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa479 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.048694:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.048704:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.048711:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:0.0:1788845046.048716:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:0.0:1788845046.048721:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1695744->1724415] (req 1695744->1724415) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.048753:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.048787:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 6 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.048800:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f825500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660407040:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845046.048918:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 7 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.048927:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 7 pid: 7983 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.048935:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660407168:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845046.048964:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.056058:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.056083:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.056145:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641811200 00000100:00100000:3.0:1788845046.056153:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641811200 00010000:00010000:3.0:1788845046.056188:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 8 pid: 7983 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.056218:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641811072 00000100:00100000:3.0:1788845046.056220:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641811072 00010000:00010000:3.0:1788845046.056229:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 7 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.056244:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 544 00000100:00100000:3.0:1788845046.056250:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d13c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641811200:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845046.056264:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845046.056288:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 8 pid: 7983 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.056300:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 8 pid: 7983 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.056328:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845046.056377:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d13c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641811200:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 126us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845046.056384:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 544 00000100:00100000:3.0:1788845046.056431:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 545 00000100:00100000:3.0:1788845046.056438:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641811072:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845046.056446:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845046.056455:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 7 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.056471:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 7 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.056485:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:3.0:1788845046.056490:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:3.0:1788845046.056511:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.056521:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.056557:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.056574:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5df800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660407296:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845046.056612:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22900 going to sleep for 11 seconds 00010000:00010000:0.0:1788845046.057052:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cae0121/0xb2187301287aefdd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa464 expref: 7 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.057069:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f825500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660407040:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845046.057080:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000005605f54/0xb2187301287aefd6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1216512->1236991) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa45d expref: 7 pid: 7983 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.057090:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660407168:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845046.057113:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.057128:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845046.057155:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641810944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8552us (8734us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.057162:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1181 00000100:00100000:1.0:1788845046.064846:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.064994:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641811328 00000100:00100000:0.0:1788845046.065007:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641811328 00002000:00100000:0.0:1788845046.065056:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1695744->1699839) req@ffff94a035740e00 x1875739641811328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.065069:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035740e00 with x1875739641811328 ext(1695744->1699839) 00002000:00010000:0.0:1788845046.065075:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035740e00 00000100:00100000:0.0:1788845046.065112:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 594 00000100:00100000:0.0:1788845046.065121:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641811328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.065390:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641811328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 270us (564us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.065399:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 594 00002000:00100000:0.0:1788845046.065406:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1695744->1699839) req@ffff94a035740e00 x1875739641811328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.065415:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035740e00 with x1875739641811328 ext(1695744->1699839) 00002000:00010000:0.0:1788845046.065419:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035740e00 00000100:00100000:3.0:1788845046.070588:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5df800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660407296:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845046.070615:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845046.070761:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641811072:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 14322us (14675us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845046.070783:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 545 00000100:00100000:1.0:1788845046.078238:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.078347:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641811456 00000100:00100000:3.0:1788845046.078357:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641811456 00002000:00100000:3.0:1788845046.078411:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1720320->1724415) req@ffff94a035740380 x1875739641811456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.078423:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035740380 with x1875739641811456 ext(1720320->1724415) 00002000:00010000:3.0:1788845046.078431:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035740380 00000100:00100000:3.0:1788845046.078465:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 595 00000100:00100000:3.0:1788845046.078477:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641811456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845046.078779:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641811456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 303us (549us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845046.078786:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 595 00002000:00100000:3.0:1788845046.078793:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1720320->1724415) req@ffff94a035740380 x1875739641811456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.078801:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a035740380 with x1875739641811456 ext(1720320->1724415) 00002000:00010000:3.0:1788845046.078805:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a035740380 00000100:00100000:2.0:1788845046.087001:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.087791:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641811584 00000100:00100000:1.0:1788845046.087805:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641811584 00000100:00100000:1.0:1788845046.087911:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1182 00000100:00100000:1.0:1788845046.087922:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641811584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845046.087946:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845046.087968:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a68d4135/0xb2187301287aeff9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa480 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.088038:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02381180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660407424:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845046.088133:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ccc0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845046.090568:0:6682:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845045 -> 1788845046 00002000:00010000:2.0:1788845046.090576:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845045 -> 1788845046 00002000:00010000:2.0:1788845046.090580:0:6682:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845045 -> 1788845046 00000100:00100000:2.0:1788845046.090604:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02381180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660407424:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845046.090651:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a68d4135/0xb2187301287aeff9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa480 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.090677:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a68d4135/0xb2187301287aeff9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa480 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.090692:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757200, rc 0) 00000100:00100000:2.0:1788845046.090753:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03446300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641811584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2832us (3762us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845046.090762:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1182 00000100:00100000:2.0:1788845046.101441:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.101539:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641811712 00000100:00100000:0.0:1788845046.101548:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641811712 00000100:00100000:0.0:1788845046.101600:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1183 00000100:00100000:0.0:1788845046.101611:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641811712:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.101812:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.101831:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287af000 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa487 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.101935:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660407552:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845046.102004:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.105712:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660407552:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845046.105758:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287af000 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa487 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.105786:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287af000 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa487 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.105801:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845046.105874:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641811712:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4264us (4449us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845046.105934:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1183 00000100:00100000:1.0:1788845046.109035:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.109179:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641811840 00000100:00100000:3.0:1788845046.109189:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641811840 00000100:00100000:3.0:1788845046.109250:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1184 00000100:00100000:3.0:1788845046.109264:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641811840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845046.109282:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845046.109300:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287af007 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa48e expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.109356:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5df800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660407680:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845046.109436:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845046.112386:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5df800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660407680:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845046.112426:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287af007 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa48e expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.112452:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002394f98/0xb2187301287af007 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa48e expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.112466:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02031800, rc 0) 00000100:00100000:3.0:1788845046.112521:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641811840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3258us (3495us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845046.112530:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1184 00000100:00100000:1.0:1788845046.119077:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.119187:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641811968 00000100:00100000:2.0:1788845046.119197:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641811968 00000100:00100000:2.0:1788845046.119263:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1185 00000100:00100000:2.0:1788845046.119274:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641811968:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.119295:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.119314:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa495 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.119351:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.119360:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(12 17) 00010000:00010000:2.0:1788845046.119365:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:2.0:1788845046.119371:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [2023424->2027519] (req 2023424->2027519) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.119406:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.119426:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 5 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.119443:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660407808:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845046.119551:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845046.126365:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.126938:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641812096 00000100:00100000:0.0:1788845046.126948:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641812096 00002000:00100000:0.0:1788845046.126998:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1695744->1724415) req@ffff94a03f777100 x1875739641812096/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.127010:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f777100 with x1875739641812096 ext(1695744->1724415) 00010000:00010000:0.0:1788845046.127019:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 6 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.127049:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 6 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845046.127057:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f777100 00000100:00100000:0.0:1788845046.127101:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 37 00000100:00100000:0.0:1788845046.127112:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641812096:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845046.130171:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660407808:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845046.130210:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [2023424->2027519] (req 2023424->2027519) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.130233:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772400, rc 0) 00000100:00100000:2.0:1788845046.130308:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641811968:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11034us (11239us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845046.130317:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1185 00000100:00100000:0.0:1788845046.131379:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641812096:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 4266us (5021us total) trans 4294967509 rc 0/0 00000100:00100000:0.0:1788845046.131391:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 37 00002000:00100000:0.0:1788845046.131402:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1695744->1724415) req@ffff94a03f777100 x1875739641812096/t4294967509(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/448 e 0 to 0 dl 1788845057 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.131412:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03f777100 with x1875739641812096 ext(1695744->1724415) 00010000:00010000:0.0:1788845046.131422:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 8 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.131441:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 8 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845046.131448:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a03f777100 00000100:00100000:2.0:1788845046.138031:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.138133:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641812224 00000100:00100000:0.0:1788845046.138144:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641812224 00010000:00010000:0.0:1788845046.138186:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 8 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.138239:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 546 00000100:00100000:0.0:1788845046.138248:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641812224:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845046.138263:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845046.138287:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 8 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.138306:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 8 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.138331:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287aeff2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1695744->1724415) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa479 expref: 8 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.138350:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:0.0:1788845046.138355:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:0.0:1788845046.138363:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.138373:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.138411:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.138426:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660407936:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845046.138521:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.144671:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660407936:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845046.144701:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845046.144767:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641812224:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6518us (6743us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.144779:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 546 00000100:00100000:3.0:1788845046.151021:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.151203:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641812352 00000100:00100000:1.0:1788845046.151214:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641812352 00002000:00100000:1.0:1788845046.151267:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2097151) req@ffff94a03d5df800 x1875739641812352/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.151279:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5df800 with x1875739641812352 ext(1048576->2097151) 00002000:00010000:1.0:1788845046.151284:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5df800 00000100:00100000:1.0:1788845046.151322:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 596 00000100:00100000:1.0:1788845046.151332:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641812352:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845046.158127:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.158225:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641812480 00000100:00100000:0.0:1788845046.158235:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641812480 00002000:00100000:0.0:1788845046.158284:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dd880 x1875739641812480/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.158297:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dd880 with x1875739641812480 00000100:00100000:0.0:1788845046.158337:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1014 00000100:00100000:0.0:1788845046.158347:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641812480:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.195370:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641812480:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 37013us (37240us total) trans 0 rc 1015808/1015808 00000100:00100000:0.0:1788845046.195385:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1014 00002000:00100000:0.0:1788845046.195409:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dd880 x1875739641812480/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/440 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.195420:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dd880 with x1875739641812480 00000100:00100000:1.0:1788845046.219640:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641812352:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 68298us (68619us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1788845046.219656:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 596 00002000:00100000:1.0:1788845046.219671:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2097151) req@ffff94a03d5df800 x1875739641812352/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/440 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.219683:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5df800 with x1875739641812352 ext(1048576->2097151) 00002000:00010000:1.0:1788845046.219690:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5df800 00000100:00100000:0.0:1788845046.231244:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.231361:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641812608 00000100:00100000:1.0:1788845046.231372:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641812608 00000100:00100000:1.0:1788845046.231453:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1602 00000100:00100000:1.0:1788845046.231463:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2369:x1875739641812608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845046.231483:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845046.231516:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287af015 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa49c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.231554:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.231599:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f774380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660408064:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845046.231947:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.243880:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000004:00010000:2.0:1788845046.248732:0:5840:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788845045 -> 1788845046 00000004:00010000:2.0:1788845046.248742:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845045 -> 1788845046 00000004:00010000:2.0:1788845046.248745:0:5840:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845045 -> 1788845046 00000100:00100000:2.0:1788845046.249045:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f774380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660408064:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845046.249070:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845046.249090:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287af015 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa49c expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.249112:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845046.249116:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845046.249121:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02757e00/0xb2187301287af015 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa49c expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.249143:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02757e00, rc 0) 00000100:00100000:2.0:1788845046.249209:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2369:x1875739641812608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 17747us (17977us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845046.249217:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1602 00000100:00100000:1.0:1788845046.251297:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641812736 00000100:00100000:1.0:1788845046.251312:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641812736 00000100:00100000:1.0:1788845046.251385:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1186 00000100:00100000:1.0:1788845046.251511:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641812736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845046.251536:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845046.251557:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4a3 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.251585:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:1.0:1788845046.251589:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:1.0:1788845046.251600:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.251626:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 8 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.251642:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02756e00, rc 0) 00000100:00100000:1.0:1788845046.251717:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641812736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 322us (7845us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845046.251725:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1186 00000100:00100000:0.0:1788845046.255573:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.255814:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641812864 00000100:00100000:3.0:1788845046.255825:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641812864 00000100:00100000:3.0:1788845046.256075:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1603 00000100:00100000:3.0:1788845046.256086:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2368:x1875739641812864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845046.256108:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845046.256135:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af023 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4aa expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.256187:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.256242:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660408192:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845046.256338:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845046.266408:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660408192:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845046.266439:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:3.0:1788845046.266459:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af023 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4aa expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.266503:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845046.266506:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845046.266512:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af023 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4aa expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.266529:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772e00, rc 0) 00000100:00100000:3.0:1788845046.266600:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2368:x1875739641812864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10515us (11040us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845046.266609:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1603 00000100:00100000:1.0:1788845046.272868:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.273819:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641812992 00000100:00100000:0.0:1788845046.273832:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641812992 00000100:00100000:0.0:1788845046.273932:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1604 00000100:00100000:0.0:1788845046.273944:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2369:x1875739641812992:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.273970:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.273991:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af02a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4b1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.274029:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.274144:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660408320:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845046.274242:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.281784:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660408320:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845046.281818:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:0.0:1788845046.281840:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af02a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4b1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.281862:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.281865:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845046.281870:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af02a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4b1 expref: 25 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.281916:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845046.281982:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2369:x1875739641812992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8040us (9127us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845046.281991:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1604 00000100:00100000:1.0:1788845046.286021:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.286206:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641813120 00000100:00100000:0.0:1788845046.286216:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641813120 00000100:00100000:0.0:1788845046.286273:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1605 00000100:00100000:0.0:1788845046.286284:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641813120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.286307:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.286328:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af031 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4b8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.286360:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.286394:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660408448:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845046.286469:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18480 going to sleep for 11 seconds 00000100:00100000:1.0:1788845046.291974:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660408448:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845046.292004:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:1.0:1788845046.292025:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af031 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4b8 expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.292048:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845046.292051:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845046.292056:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af031 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4b8 expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.292071:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007000, rc 0) 00000100:00100000:1.0:1788845046.292139:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2370:x1875739641813120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5855us (6128us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845046.292148:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1605 00000100:00100000:1.0:1788845046.301511:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.301588:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641813248 00000100:00100000:2.0:1788845046.301598:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641813248 00000100:00100000:2.0:1788845046.301668:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1606 00000100:00100000:2.0:1788845046.301677:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2368:x1875739641813248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.301696:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.301713:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af038 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4bf expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.301748:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.301792:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660408576:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845046.301875:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.303923:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660408576:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845046.303953:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845046.303976:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af038 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4bf expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.303999:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845046.304002:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845046.304007:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af038 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4bf expref: 25 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.304023:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865c00, rc 0) 00000100:00100000:2.0:1788845046.304088:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:2368:x1875739641813248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2410us (2584us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845046.304096:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1606 00000100:00100000:1.0:1788845046.309652:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.309792:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641813376 00000100:00100000:3.0:1788845046.309803:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641813376 00000100:00100000:3.0:1788845046.309866:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1607 00000100:00100000:3.0:1788845046.309877:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+24:14891:x1875739641813376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845046.309927:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845046.309952:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4c6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.310017:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.310034:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4c6 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.310081:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.310099:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 26 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.310112:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660408704:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845046.310192:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.312678:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660408704:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845046.313084:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4c6 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.313114:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.313118:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845046.313124:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4c6 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.313134:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772e00, rc 0) 00000100:00100000:0.0:1788845046.313237:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641813376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3360us (3594us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.313247:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1607 00000100:00100000:1.0:1788845046.321011:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.321130:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641813504 00000100:00100000:0.0:1788845046.321141:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641813504 00010000:00010000:0.0:1788845046.321181:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.321224:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 547 00000100:00100000:0.0:1788845046.321235:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641813504:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845046.321255:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845046.321283:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.321302:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.321329:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030000/0xb2187301287aef97 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa41e expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.321354:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.321359:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4c6 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.321370:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4c6 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.321379:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4c6 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.321432:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4c6 expref: 24 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.321446:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660408832:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845046.321528:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.331505:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660408832:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845046.331528:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845046.331579:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641813504:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10346us (10579us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.331589:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 547 00000100:00100000:1.0:1788845046.342801:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.342913:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641813632 00000100:00100000:0.0:1788845046.342923:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641813632 00002000:00100000:0.0:1788845046.342975:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3b80 x1875739641813632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.342988:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3b80 with x1875739641813632 00000100:00100000:0.0:1788845046.343032:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1015 00000100:00100000:0.0:1788845046.343042:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641813632:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.346033:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.346430:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641813760 00000100:00100000:3.0:1788845046.346439:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641813760 00002000:00100000:3.0:1788845046.346545:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2097151) req@ffff949f033f0000 x1875739641813760/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.346558:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f0000 with x1875739641813760 ext(1048576->2097151) 00002000:00010000:3.0:1788845046.346563:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f0000 00000100:00100000:3.0:1788845046.346643:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 597 00000100:00100000:3.0:1788845046.346653:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641813760:12345-192.168.201.29@tcp:3:fsx.0 00010000:00010000:1.0:1788845046.354107:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845046.390405:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2368:x1875739641813632:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 47353us (47600us total) trans 0 rc 1015808/1015808 00000100:00100000:0.0:1788845046.390420:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1015 00002000:00100000:0.0:1788845046.390435:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3b80 x1875739641813632/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/440 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.390445:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3b80 with x1875739641813632 00000100:00100000:1.0:1788845046.397090:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.400264:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641813760:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 53602us (54230us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1788845046.400279:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 597 00002000:00100000:2.0:1788845046.400292:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2097151) req@ffff949f033f0000 x1875739641813760/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/440 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.400304:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f033f0000 with x1875739641813760 ext(1048576->2097151) 00002000:00010000:2.0:1788845046.400310:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f033f0000 00000100:00100000:0.0:1788845046.401035:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641813888 00000100:00100000:0.0:1788845046.401044:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641813888 00000100:00100000:0.0:1788845046.401097:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1608 00000100:00100000:0.0:1788845046.401106:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a04a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641813888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.401125:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.401157:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4cd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.401193:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 27 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845046.401204:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:0.0:1788845046.401211:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 27 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.401224:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.401227:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845046.401231:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845046.401272:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a04a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641813888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 167us (4192us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.401279:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1608 00000100:00100000:1.0:1788845046.412372:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.412512:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641814016 00000100:00100000:0.0:1788845046.412522:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641814016 00000100:00100000:0.0:1788845046.412585:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1187 00000100:00100000:0.0:1788845046.412595:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a04000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:14891:x1875739641814016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.412624:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.412644:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4d4 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.412677:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.412790:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.412798:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:0.0:1788845046.412802:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:0.0:1788845046.412807:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 5/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [1433600->1437695] (req 1433600->1437695) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 8 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.412840:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.412858:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 7 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.412874:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660408960:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845046.415032:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 8 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.415053:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 8 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.415069:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a06a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660409088:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845046.415099:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845046.427973:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.427995:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.428218:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641814144 00000100:00100000:3.0:1788845046.428226:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641814144 00010000:00010000:3.0:1788845046.428262:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 8 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.428300:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641814272 00000100:00100000:3.0:1788845046.428304:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641814272 00010000:00010000:3.0:1788845046.428318:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 9 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.428342:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 548 00000100:00100000:3.0:1788845046.428349:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a05880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641814144:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845046.428361:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845046.428379:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 8 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.428392:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 8 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.428412:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845046.428450:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a05880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641814144:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 102us (487us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845046.428455:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 548 00000100:00100000:3.0:1788845046.428478:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 549 00000100:00100000:3.0:1788845046.428482:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a07100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2370:x1875739641814272:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845046.428494:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845046.428500:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 9 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.428507:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 9 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.428516:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:3.0:1788845046.428520:0:5825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(13 17) 00010000:00010000:3.0:1788845046.428526:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.428532:0:5825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.428562:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 9 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.428572:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02365c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660409216:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845046.428607:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00010000:00010000:2.0:1788845046.428717:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a75f00a/0xb2187301287af01c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4a3 expref: 7 pid: 6682 timeout: 535 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.428733:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660408960:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845046.428746:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ace2c2c7/0xb2187301287af00e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2027519) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa495 expref: 9 pid: 6685 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.428756:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a06a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660409088:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845046.428780:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 8 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.428798:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006800, rc 0) 00000100:00100000:2.0:1788845046.428822:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a04000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:14891:x1875739641814016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 16229us (16462us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845046.428829:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1187 00000100:00100000:3.0:1788845046.431784:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02365c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5825:1875739660409216:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845046.431809:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845046.431950:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a07100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814272:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3464us (3951us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845046.431961:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 549 00000100:00100000:1.0:1788845046.445162:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.445241:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641814400 00000100:00100000:0.0:1788845046.445251:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641814400 00002000:00100000:0.0:1788845046.445302:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1433600->1437695) req@ffff949f03a05f80 x1875739641814400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.445313:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a05f80 with x1875739641814400 ext(1433600->1437695) 00002000:00010000:0.0:1788845046.445319:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a05f80 00000100:00100000:0.0:1788845046.445356:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 598 00000100:00100000:0.0:1788845046.445365:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a05f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814400:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.445595:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a05f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814400:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 230us (442us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.445603:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 598 00002000:00100000:0.0:1788845046.445610:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1433600->1437695) req@ffff949f03a05f80 x1875739641814400/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.445629:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a05f80 with x1875739641814400 ext(1433600->1437695) 00002000:00010000:0.0:1788845046.445633:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a05f80 00000100:00100000:1.0:1788845046.451858:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.452204:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641814528 00000100:00100000:0.0:1788845046.452213:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641814528 00002000:00100000:0.0:1788845046.452259:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1437696->1441791) req@ffff949f03a04380 x1875739641814528/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.452267:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a04380 with x1875739641814528 ext(1437696->1441791) 00002000:00010000:0.0:1788845046.452272:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a04380 00000100:00100000:0.0:1788845046.452299:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 599 00000100:00100000:0.0:1788845046.452306:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a04380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814528:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.452523:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a04380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814528:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 215us (672us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.452530:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 599 00002000:00100000:0.0:1788845046.452538:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1437696->1441791) req@ffff949f03a04380 x1875739641814528/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.452547:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a04380 with x1875739641814528 ext(1437696->1441791) 00002000:00010000:0.0:1788845046.452550:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a04380 00000100:00100000:2.0:1788845046.457000:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.457085:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641814656 00000100:00100000:0.0:1788845046.457095:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641814656 00002000:00100000:0.0:1788845046.457145:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1441792->1445887) req@ffff949f02361180 x1875739641814656/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.457156:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02361180 with x1875739641814656 ext(1441792->1445887) 00002000:00010000:0.0:1788845046.457163:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02361180 00000100:00100000:0.0:1788845046.457246:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 600 00000100:00100000:0.0:1788845046.457255:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814656:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.457494:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814656:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 239us (501us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.457502:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 600 00002000:00100000:0.0:1788845046.457509:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1441792->1445887) req@ffff949f02361180 x1875739641814656/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.457517:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02361180 with x1875739641814656 ext(1441792->1445887) 00002000:00010000:0.0:1788845046.457520:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02361180 00000100:00100000:1.0:1788845046.461473:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.461702:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641814784 00000100:00100000:0.0:1788845046.461714:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641814784 00002000:00100000:0.0:1788845046.461763:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1449983) req@ffff949f03a07800 x1875739641814784/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.461775:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a07800 with x1875739641814784 ext(1445888->1449983) 00002000:00010000:0.0:1788845046.461781:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a07800 00000100:00100000:0.0:1788845046.461818:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 601 00000100:00100000:0.0:1788845046.461828:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a07800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814784:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.462109:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a07800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641814784:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (647us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.462118:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 601 00002000:00100000:0.0:1788845046.462125:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1445888->1449983) req@ffff949f03a07800 x1875739641814784/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.462134:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a07800 with x1875739641814784 ext(1445888->1449983) 00002000:00010000:0.0:1788845046.462137:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a07800 00000100:00100000:3.0:1788845046.466847:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.467020:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641814912 00000100:00100000:1.0:1788845046.467030:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641814912 00002000:00100000:1.0:1788845046.467078:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1449984->1454079) req@ffff949f02365c00 x1875739641814912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.467090:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365c00 with x1875739641814912 ext(1449984->1454079) 00002000:00010000:1.0:1788845046.467095:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365c00 00000100:00100000:1.0:1788845046.467139:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 602 00000100:00100000:1.0:1788845046.467149:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641814912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.467391:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02365c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641814912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 242us (551us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.467403:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 602 00002000:00100000:1.0:1788845046.467410:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1449984->1454079) req@ffff949f02365c00 x1875739641814912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.467418:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02365c00 with x1875739641814912 ext(1449984->1454079) 00002000:00010000:1.0:1788845046.467421:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02365c00 00000100:00100000:1.0:1788845046.471567:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.471766:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641815040 00000100:00100000:0.0:1788845046.471778:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641815040 00002000:00100000:0.0:1788845046.471825:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1454080->1458175) req@ffff949f03a05c00 x1875739641815040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.471836:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a05c00 with x1875739641815040 ext(1454080->1458175) 00002000:00010000:0.0:1788845046.471841:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a05c00 00000100:00100000:0.0:1788845046.471877:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 603 00000100:00100000:0.0:1788845046.471923:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a05c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641815040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.472186:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a05c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641815040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 299us (627us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.472194:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 603 00002000:00100000:0.0:1788845046.472202:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1454080->1458175) req@ffff949f03a05c00 x1875739641815040/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.472210:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a05c00 with x1875739641815040 ext(1454080->1458175) 00002000:00010000:0.0:1788845046.472213:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a05c00 00000100:00100000:3.0:1788845046.476210:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.476332:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641815168 00000100:00100000:1.0:1788845046.476343:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641815168 00002000:00100000:1.0:1788845046.476401:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1458176->1462271) req@ffff949f02364700 x1875739641815168/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.476413:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02364700 with x1875739641815168 ext(1458176->1462271) 00002000:00010000:1.0:1788845046.476418:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02364700 00000100:00100000:1.0:1788845046.476456:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 604 00000100:00100000:1.0:1788845046.476468:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641815168:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.476759:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641815168:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 292us (560us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.476767:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 604 00002000:00100000:1.0:1788845046.476773:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1458176->1462271) req@ffff949f02364700 x1875739641815168/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.476781:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f02364700 with x1875739641815168 ext(1458176->1462271) 00002000:00010000:1.0:1788845046.476785:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f02364700 00000100:00100000:3.0:1788845046.480830:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.480942:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641815296 00000100:00100000:0.0:1788845046.480952:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641815296 00002000:00100000:0.0:1788845046.480997:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1433600->1462271) req@ffff94a010480a80 x1875739641815296/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.481011:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010480a80 with x1875739641815296 ext(1433600->1462271) 00002000:00010000:0.0:1788845046.481017:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010480a80 00000100:00100000:0.0:1788845046.481049:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 605 00000100:00100000:0.0:1788845046.481059:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2367:x1875739641815296:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:3.0:1788845046.482214:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845046.482233:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660409344:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845046.482283:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845046.482732:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660409344 00000100:00100000:0.0:1788845046.482739:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660409344 00000100:00100000:0.0:1788845046.482783:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 85 00000100:00100000:0.0:1788845046.482791:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660409344:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788845046.482812:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 158400512 pending 3407872 free 3859808256 avail 3849322496 00000100:00100000:0.0:1788845046.482866:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660409344:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (582us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.482874:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 85 00000100:00100000:1.0:1788845046.482990:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660409344:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845046.486275:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+9:2367:x1875739641815296:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5206us (5445us total) trans 4294967510 rc 0/0 00000100:00100000:0.0:1788845046.486288:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 605 00002000:00100000:0.0:1788845046.486300:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1433600->1462271) req@ffff94a010480a80 x1875739641815296/t4294967510(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/448 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.486311:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a010480a80 with x1875739641815296 ext(1433600->1462271) 00002000:00010000:0.0:1788845046.486316:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a010480a80 00000100:00100000:3.0:1788845046.496300:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.496574:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641815424 00000100:00100000:1.0:1788845046.496587:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641815424 00000100:00100000:1.0:1788845046.496671:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1188 00000100:00100000:1.0:1788845046.496683:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+9:2368:x1875739641815424:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:3.0:1788845046.528749:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967510 is committed 00000001:00080000:3.0:1788845046.528768:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967509 is committed 00000100:00100000:1.0:1788845046.529023:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2368:x1875739641815424:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 32336us (32728us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845046.529034:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1188 00000100:00100000:1.0:1788845046.536015:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.536119:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641815552 00000100:00100000:0.0:1788845046.536196:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641815552 00000100:00100000:0.0:1788845046.536304:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1609 00000100:00100000:0.0:1788845046.536315:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a06300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641815552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.536335:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.536338:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788845046.536381:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af03f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4c6 expref: 25 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.536400:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.536404:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845046.536418:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4db expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.536442:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.536452:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.536623:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 26 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.536645:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 26 pid: 5841 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.536660:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660409472:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845046.536746:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c183c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.539914:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.540122:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660409472:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845046.540164:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.540184:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845046.540187:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845046.540191:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 25 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.540202:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:1.0:1788845046.540277:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a06300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:14891:x1875739641815552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3963us (4271us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845046.540285:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1609 00000100:00100000:3.0:1788845046.540393:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641815680 00000100:00100000:3.0:1788845046.540400:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641815680 00010000:00010000:3.0:1788845046.540431:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 27 pid: 5841 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.540457:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 550 00000100:00100000:3.0:1788845046.540465:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641815680:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845046.540477:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845046.540510:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 27 pid: 5841 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.540526:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 27 pid: 5841 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.540546:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af046 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4cd expref: 27 pid: 5841 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.540559:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845046.540563:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.540573:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.540583:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.540622:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.540633:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660409600:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845046.540681:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.544314:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.544670:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660409600:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845046.544693:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845046.544751:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+26:2369:x1875739641815680:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4285us (4870us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845046.544762:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 550 00000100:00100000:3.0:1788845046.544803:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641815808 00000100:00100000:3.0:1788845046.544811:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641815808 00002000:00100000:3.0:1788845046.544850:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824a80 x1875739641815808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.544860:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824a80 with x1875739641815808 00000100:00100000:3.0:1788845046.544906:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1016 00000100:00100000:3.0:1788845046.544916:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641815808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845046.545161:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2367:x1875739641815808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 246us (855us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845046.545168:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1016 00002000:00100000:3.0:1788845046.545175:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f824a80 x1875739641815808/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.545182:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f824a80 with x1875739641815808 00000100:00100000:0.0:1788845046.549828:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.550011:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641815936 00000100:00100000:2.0:1788845046.550021:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641815936 00002000:00100000:1.0:1788845046.550136:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b8000 x1875739641815936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.550149:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b8000 with x1875739641815936 00000100:00100000:1.0:1788845046.550203:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1017 00000100:00100000:1.0:1788845046.550212:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641815936:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.550467:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641815936:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (641us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.550475:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1017 00002000:00100000:1.0:1788845046.550482:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0359b8000 x1875739641815936/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.550927:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0359b8000 with x1875739641815936 00000100:00100000:0.0:1788845046.554123:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.554287:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641816064 00000100:00100000:2.0:1788845046.554297:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641816064 00000100:00100000:2.0:1788845046.554385:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1189 00000100:00100000:2.0:1788845046.554395:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641816064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.554413:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.554429:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af05b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4e2 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.554484:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660409728:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845046.554601:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00002000:00010000:0.0:1788845046.556607:0:6685:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 269 -> 383 00000100:00100000:0.0:1788845046.556643:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660409728:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845046.556678:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af05b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4e2 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.556702:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af05b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4e2 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.556715:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00000100:00100000:0.0:1788845046.556812:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641816064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2418us (2697us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845046.556821:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1189 00000100:00100000:1.0:1788845046.560046:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.560197:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641816192 00000100:00100000:2.0:1788845046.560207:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641816192 00000100:00100000:2.0:1788845046.560343:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1190 00000100:00100000:2.0:1788845046.560354:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a05500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641816192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.560373:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.560391:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af062 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4e9 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.560451:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660409856:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845046.560564:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.562949:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660409856:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845046.562989:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af062 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4e9 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.563013:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af062 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4e9 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.563026:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845046.563179:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a05500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641816192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2825us (3141us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845046.563188:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1190 00000100:00100000:0.0:1788845046.566439:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.566582:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641816320 00000100:00100000:2.0:1788845046.566592:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641816320 00000100:00100000:2.0:1788845046.566697:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1610 00000100:00100000:2.0:1788845046.566708:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2370:x1875739641816320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.566730:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.566754:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af069 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4f0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.566793:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.566838:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02363100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660409984:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845046.566958:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.569004:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02363100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660409984:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845046.569032:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1011804 00010000:00010000:2.0:1788845046.569052:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af069 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f0 expref: 27 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.569073:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845046.569077:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845046.569082:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af069 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f0 expref: 27 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.569097:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865400, rc 0) 00000100:00100000:2.0:1788845046.569164:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+26:2370:x1875739641816320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2457us (2732us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845046.569173:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1610 00000100:00100000:3.0:1788845046.571731:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.572351:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641816448 00000100:00100000:1.0:1788845046.572361:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641816448 00000100:00100000:1.0:1788845046.572417:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1611 00000100:00100000:1.0:1788845046.572426:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+26:14891:x1875739641816448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845046.572446:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845046.572465:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4f7 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.572554:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.572564:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 27 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.572596:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 24 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.572625:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 24 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.572644:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a05180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660410112:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845046.572724:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00562ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845046.575557:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a05180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660410112:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845046.575593:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 27 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.575622:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845046.575626:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845046.575631:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 27 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.575642:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031200, rc 0) 00000100:00100000:1.0:1788845046.575691:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:14891:x1875739641816448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3265us (3968us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845046.575699:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1611 00000100:00100000:1.0:1788845046.575792:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.576501:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641816576 00000100:00100000:3.0:1788845046.576510:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641816576 00002000:00100000:3.0:1788845046.576556:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a05180 x1875739641816576/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.576566:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a05180 with x1875739641816576 00010000:00010000:3.0:1788845046.576573:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 25 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.576596:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 25 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845046.576603:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f03a05180 00000100:00100000:3.0:1788845046.576647:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 39 00000100:00100000:3.0:1788845046.576656:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a05180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+25:2370:x1875739641816576:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:1.0:1788845046.579791:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1011804 -> 1011804 00000004:00010000:1.0:1788845046.579801:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00000004:00010000:1.0:1788845046.579804:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845046 -> 1788845043 00000004:00010000:1.0:1788845046.579807:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00000004:00010000:1.0:1788845046.579809:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:1.0:1788845046.579909:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a05180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2370:x1875739641816576:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3223us (4085us total) trans 4294967862 rc 0/0 00000100:00100000:1.0:1788845046.579922:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 39 00002000:00100000:1.0:1788845046.579935:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a05180 x1875739641816576/t4294967862(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/448 e 0 to 0 dl 1788845057 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.579944:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a05180 with x1875739641816576 00010000:00010000:1.0:1788845046.579953:0:5852:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.579971:0:5852:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845046.579980:0:5852:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f03a05180 00000100:00100000:1.0:1788845046.582653:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.582739:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641816704 00000100:00100000:0.0:1788845046.582748:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641816704 00010000:00010000:0.0:1788845046.582781:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.582816:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 551 00000100:00100000:0.0:1788845046.582826:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d10850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+27:2370:x1875739641816704:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845046.582841:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845046.582869:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.582912:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.582934:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af054 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4db expref: 27 pid: 11036 timeout: 535 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.582950:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.582956:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.582968:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.582977:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.583015:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 26 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.583029:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660410240:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845046.583216:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18840 going to sleep for 11 seconds 00000100:00100000:3.0:1788845046.585352:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.585370:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660410240:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845046.585392:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845046.585490:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641816832 00000100:00100000:2.0:1788845046.585497:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641816832 00002000:00100000:2.0:1788845046.585545:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999500 x1875739641816832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.585557:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999500 with x1875739641816832 00000100:00100000:2.0:1788845046.585595:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1018 00000100:00100000:2.0:1788845046.585605:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641816832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.586019:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d10850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+26:2370:x1875739641816704:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3191us (3372us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.586028:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 551 00000100:00100000:2.0:1788845046.586185:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641816832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 581us (840us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845046.586193:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1018 00002000:00100000:2.0:1788845046.586200:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999500 x1875739641816832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.586208:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999500 with x1875739641816832 00000100:00100000:3.0:1788845046.588455:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.588579:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641816960 00000100:00100000:1.0:1788845046.588587:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641816960 00002000:00100000:1.0:1788845046.588638:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999180 x1875739641816960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.588648:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999180 with x1875739641816960 00000100:00100000:1.0:1788845046.588787:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1019 00000100:00100000:1.0:1788845046.588796:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641816960:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.588999:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641816960:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 204us (550us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.589004:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1019 00002000:00100000:1.0:1788845046.589009:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999180 x1875739641816960/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.589014:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999180 with x1875739641816960 00000100:00100000:3.0:1788845046.591089:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.591308:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641817088 00000100:00100000:1.0:1788845046.591317:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641817088 00002000:00100000:1.0:1788845046.591364:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399b480 x1875739641817088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.591373:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399b480 with x1875739641817088 00000100:00100000:1.0:1788845046.591403:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1020 00000100:00100000:1.0:1788845046.591413:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641817088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.591973:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641817088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 560us (892us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.591979:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1020 00002000:00100000:1.0:1788845046.591985:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399b480 x1875739641817088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.591990:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399b480 with x1875739641817088 00000100:00100000:2.0:1788845046.594046:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.594136:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641817216 00000100:00100000:3.0:1788845046.594146:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641817216 00002000:00100000:3.0:1788845046.594191:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02363100 x1875739641817216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.594203:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02363100 with x1875739641817216 00000100:00100000:3.0:1788845046.594239:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1021 00000100:00100000:3.0:1788845046.594249:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845046.594453:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817216:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 206us (412us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845046.594461:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1021 00002000:00100000:3.0:1788845046.594468:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02363100 x1875739641817216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.594476:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02363100 with x1875739641817216 00000100:00100000:3.0:1788845046.596631:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.596731:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641817344 00000100:00100000:2.0:1788845046.596742:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641817344 00002000:00100000:2.0:1788845046.596787:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399a680 x1875739641817344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.596798:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399a680 with x1875739641817344 00000100:00100000:2.0:1788845046.596840:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1022 00000100:00100000:2.0:1788845046.596850:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817344:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845046.597142:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817344:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 294us (519us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845046.597151:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1022 00002000:00100000:2.0:1788845046.597158:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399a680 x1875739641817344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.597166:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399a680 with x1875739641817344 00000100:00100000:1.0:1788845046.599716:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.599806:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641817472 00000100:00100000:0.0:1788845046.599815:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641817472 00002000:00100000:0.0:1788845046.599857:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a07480 x1875739641817472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.599867:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a07480 with x1875739641817472 00000100:00100000:3.0:1788845046.599971:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1023 00000100:00100000:3.0:1788845046.599984:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a07480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817472:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845046.600233:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a07480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817472:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (528us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845046.600239:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1023 00002000:00100000:3.0:1788845046.600243:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a07480 x1875739641817472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.600249:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a07480 with x1875739641817472 00000100:00100000:3.0:1788845046.602994:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.603258:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641817600 00000100:00100000:2.0:1788845046.603269:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641817600 00002000:00100000:2.0:1788845046.603318:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399bb80 x1875739641817600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.603330:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399bb80 with x1875739641817600 00000100:00100000:2.0:1788845046.603368:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1024 00000100:00100000:2.0:1788845046.603378:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845046.603781:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 402us (795us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845046.603789:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1024 00002000:00100000:2.0:1788845046.603797:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399bb80 x1875739641817600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.603806:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399bb80 with x1875739641817600 00000100:00100000:1.0:1788845046.606734:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.606788:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641817728 00000100:00100000:3.0:1788845046.606797:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641817728 00002000:00100000:3.0:1788845046.606839:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a06680 x1875739641817728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.606848:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a06680 with x1875739641817728 00000100:00100000:3.0:1788845046.606943:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1025 00000100:00100000:3.0:1788845046.606954:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a06680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817728:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.607255:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a06680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817728:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 301us (526us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.607263:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1025 00002000:00100000:1.0:1788845046.607271:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a06680 x1875739641817728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.607278:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a06680 with x1875739641817728 00000100:00100000:1.0:1788845046.609675:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.609871:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641817856 00000100:00100000:3.0:1788845046.609879:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641817856 00002000:00100000:3.0:1788845046.609957:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a07b80 x1875739641817856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.609966:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a07b80 with x1875739641817856 00000100:00100000:3.0:1788845046.609995:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1026 00000100:00100000:3.0:1788845046.610005:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a07b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641817856:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845046.610270:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a07b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641817856:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (613us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845046.610277:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1026 00002000:00100000:3.0:1788845046.610282:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a07b80 x1875739641817856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.610287:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a07b80 with x1875739641817856 00000100:00100000:3.0:1788845046.612593:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.612690:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641817984 00000100:00100000:0.0:1788845046.612698:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641817984 00002000:00100000:0.0:1788845046.612734:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998700 x1875739641817984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.612741:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998700 with x1875739641817984 00000100:00100000:0.0:1788845046.612769:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1027 00000100:00100000:0.0:1788845046.612775:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.613023:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641817984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 246us (436us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.613030:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1027 00002000:00100000:0.0:1788845046.613037:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998700 x1875739641817984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.613050:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998700 with x1875739641817984 00000100:00100000:1.0:1788845046.615746:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.615916:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641818112 00000100:00100000:2.0:1788845046.615927:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641818112 00002000:00100000:2.0:1788845046.615978:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a04700 x1875739641818112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.615990:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a04700 with x1875739641818112 00000100:00100000:2.0:1788845046.616024:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1028 00000100:00100000:2.0:1788845046.616319:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a04700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641818112:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845046.616649:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a04700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641818112:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 332us (911us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845046.616657:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1028 00002000:00100000:2.0:1788845046.616731:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a04700 x1875739641818112/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.616739:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a04700 with x1875739641818112 00000100:00100000:3.0:1788845046.620143:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.620314:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641818240 00000100:00100000:0.0:1788845046.620326:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641818240 00002000:00100000:0.0:1788845046.620385:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399ad80 x1875739641818240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.620396:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399ad80 with x1875739641818240 00000100:00100000:0.0:1788845046.620514:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1029 00000100:00100000:0.0:1788845046.620524:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641818240:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.620829:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+27:2370:x1875739641818240:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 306us (700us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.620837:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1029 00002000:00100000:0.0:1788845046.620845:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399ad80 x1875739641818240/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.620853:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399ad80 with x1875739641818240 00000100:00100000:3.0:1788845046.634156:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.638039:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641818368 00000100:00100000:1.0:1788845046.638059:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641818368 00002000:00100000:1.0:1788845046.638126:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998e00 x1875739641818368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.638143:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998e00 with x1875739641818368 00000100:00100000:2.0:1788845046.640151:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1030 00000100:00100000:2.0:1788845046.640174:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641818368:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845046.644370:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2369:x1875739641818368:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 4194us (10214us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845046.644389:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1030 00002000:00100000:2.0:1788845046.644405:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998e00 x1875739641818368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.644416:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998e00 with x1875739641818368 00000100:00100000:3.0:1788845046.670702:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.671221:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641818496 00000100:00100000:2.0:1788845046.671233:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641818496 00002000:00100000:2.0:1788845046.671284:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998a80 x1875739641818496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.671295:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998a80 with x1875739641818496 00000100:00100000:2.0:1788845046.671336:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1031 00000100:00100000:2.0:1788845046.671346:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2367:x1875739641818496:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845046.671617:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2367:x1875739641818496:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 263us (925us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845046.671625:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1031 00002000:00100000:2.0:1788845046.671632:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998a80 x1875739641818496/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.671639:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998a80 with x1875739641818496 00000100:00100000:3.0:1788845046.675659:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.675762:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641818624 00000100:00100000:1.0:1788845046.675837:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641818624 00002000:00100000:1.0:1788845046.675908:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998000 x1875739641818624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.675919:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998000 with x1875739641818624 00000100:00100000:1.0:1788845046.675957:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1032 00000100:00100000:1.0:1788845046.675968:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2368:x1875739641818624:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.676339:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+27:2368:x1875739641818624:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 371us (687us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.676348:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1032 00002000:00100000:1.0:1788845046.676356:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998000 x1875739641818624/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.676364:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998000 with x1875739641818624 00000100:00100000:3.0:1788845046.683229:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.683338:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641818752 00000100:00100000:2.0:1788845046.683347:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641818752 00002000:00100000:2.0:1788845046.683392:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399aa00 x1875739641818752/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.683405:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399aa00 with x1875739641818752 00000100:00100000:2.0:1788845046.683447:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1033 00000100:00100000:2.0:1788845046.683456:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+27:2368:x1875739641818752:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:0.0:1788845046.687360:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1011804 -> 1011804 00000004:00010000:0.0:1788845046.687369:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00000004:00010000:0.0:1788845046.687372:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:0.0:1788845046.687375:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00000004:00010000:0.0:1788845046.687377:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845046.688739:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+29:2368:x1875739641818752:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5274us (5510us total) trans 4294967863 rc 0/0 00000100:00100000:0.0:1788845046.688752:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1033 00002000:00100000:0.0:1788845046.688767:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399aa00 x1875739641818752/t4294967863(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/448 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.688782:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399aa00 with x1875739641818752 00000100:00100000:2.0:1788845046.691578:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.691683:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641818880 00000100:00100000:3.0:1788845046.691693:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641818880 00000100:00100000:3.0:1788845046.691777:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1612 00000100:00100000:3.0:1788845046.691788:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+29:2369:x1875739641818880:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:0.0:1788845046.777173:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967863 is committed 00000001:00080000:0.0:1788845046.777203:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967847 is committed 00000001:00080000:0.0:1788845046.777215:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967862 is committed 00000100:00100000:2.0:1788845046.777827:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cedf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641818880:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 86033us (86333us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845046.777842:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1612 00000100:00100000:2.0:1788845046.782518:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.782827:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641819008 00000100:00100000:1.0:1788845046.782838:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641819008 00002000:00100000:1.0:1788845046.782936:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1568768->1572863) req@ffff949f01cee680 x1875739641819008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.782949:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cee680 with x1875739641819008 ext(1568768->1572863) 00002000:00010000:1.0:1788845046.782956:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cee680 00000100:00100000:1.0:1788845046.782984:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 606 00000100:00100000:1.0:1788845046.782994:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641819008:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.783399:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641819008:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 404us (889us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.783408:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 606 00002000:00100000:1.0:1788845046.783416:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1568768->1572863) req@ffff949f01cee680 x1875739641819008/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.783424:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f01cee680 with x1875739641819008 ext(1568768->1572863) 00002000:00010000:1.0:1788845046.783428:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f01cee680 00000100:00100000:1.0:1788845046.787328:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.787419:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641819136 00000100:00100000:2.0:1788845046.787430:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641819136 00002000:00100000:2.0:1788845046.787479:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1572864->1576959) req@ffff949f03a06d80 x1875739641819136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.787553:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a06d80 with x1875739641819136 ext(1572864->1576959) 00002000:00010000:2.0:1788845046.787559:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a06d80 00000100:00100000:2.0:1788845046.787653:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 607 00000100:00100000:2.0:1788845046.787663:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a06d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641819136:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845046.787916:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a06d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641819136:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (597us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845046.787924:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 607 00002000:00100000:2.0:1788845046.787942:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1572864->1576959) req@ffff949f03a06d80 x1875739641819136/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.787951:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a06d80 with x1875739641819136 ext(1572864->1576959) 00002000:00010000:2.0:1788845046.787955:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a06d80 00000100:00100000:2.0:1788845046.791660:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.791760:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641819264 00000100:00100000:0.0:1788845046.791771:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641819264 00002000:00100000:0.0:1788845046.791823:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1576960->1581055) req@ffff94a036083b80 x1875739641819264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.791836:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036083b80 with x1875739641819264 ext(1576960->1581055) 00002000:00010000:0.0:1788845046.791842:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036083b80 00000100:00100000:0.0:1788845046.791880:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 608 00000100:00100000:0.0:1788845046.791924:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641819264:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.792357:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641819264:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 434us (705us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.792365:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 608 00002000:00100000:0.0:1788845046.792373:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1576960->1581055) req@ffff94a036083b80 x1875739641819264/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.792381:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a036083b80 with x1875739641819264 ext(1576960->1581055) 00002000:00010000:0.0:1788845046.792385:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a036083b80 00000100:00100000:3.0:1788845046.795580:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.796188:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641819392 00000100:00100000:0.0:1788845046.796199:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641819392 00002000:00100000:0.0:1788845046.796251:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1581056->1585151) req@ffff949f03999880 x1875739641819392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.796263:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03999880 with x1875739641819392 ext(1581056->1585151) 00002000:00010000:0.0:1788845046.796269:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03999880 00000100:00100000:0.0:1788845046.796308:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 609 00000100:00100000:0.0:1788845046.796318:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641819392:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.796549:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641819392:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 232us (979us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.796554:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 609 00002000:00100000:0.0:1788845046.796560:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1581056->1585151) req@ffff949f03999880 x1875739641819392/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.796566:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03999880 with x1875739641819392 ext(1581056->1585151) 00002000:00010000:0.0:1788845046.796568:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03999880 00000100:00100000:1.0:1788845046.799648:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.799771:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641819520 00000100:00100000:3.0:1788845046.799781:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641819520 00002000:00100000:3.0:1788845046.799869:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1585152->1589247) req@ffff949f03a04e00 x1875739641819520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845046.799877:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a04e00 with x1875739641819520 ext(1585152->1589247) 00002000:00010000:3.0:1788845046.799976:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a04e00 00000100:00100000:2.0:1788845046.800120:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 610 00000100:00100000:2.0:1788845046.800144:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a04e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641819520:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845046.800336:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a04e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641819520:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 196us (702us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845046.800342:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 610 00002000:00100000:2.0:1788845046.800346:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1585152->1589247) req@ffff949f03a04e00 x1875739641819520/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.800352:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a04e00 with x1875739641819520 ext(1585152->1589247) 00002000:00010000:2.0:1788845046.800355:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a04e00 00000100:00100000:1.0:1788845046.802466:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.802731:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641819648 00000100:00100000:0.0:1788845046.802742:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641819648 00002000:00100000:0.0:1788845046.802790:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1589248->1593343) req@ffff949f03a05880 x1875739641819648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.802801:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a05880 with x1875739641819648 ext(1589248->1593343) 00002000:00010000:0.0:1788845046.802807:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a05880 00000100:00100000:0.0:1788845046.802841:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 611 00000100:00100000:0.0:1788845046.802852:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a05880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641819648:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.803104:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a05880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641819648:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (646us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.803112:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 611 00002000:00100000:0.0:1788845046.803118:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1589248->1593343) req@ffff949f03a05880 x1875739641819648/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.803127:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a05880 with x1875739641819648 ext(1589248->1593343) 00002000:00010000:0.0:1788845046.803131:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a05880 00000100:00100000:3.0:1788845046.808658:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.808765:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641819776 00000100:00100000:0.0:1788845046.808775:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641819776 00002000:00100000:0.0:1788845046.808824:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1597439) req@ffff949f0399b100 x1875739641819776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.808835:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399b100 with x1875739641819776 ext(1593344->1597439) 00002000:00010000:0.0:1788845046.808841:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399b100 00000100:00100000:0.0:1788845046.808878:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 612 00000100:00100000:0.0:1788845046.808917:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641819776:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845046.809166:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641819776:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 249us (516us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845046.809173:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 612 00002000:00100000:0.0:1788845046.809180:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1593344->1597439) req@ffff949f0399b100 x1875739641819776/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.809188:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399b100 with x1875739641819776 ext(1593344->1597439) 00002000:00010000:0.0:1788845046.809191:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399b100 00000100:00100000:1.0:1788845046.819866:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.819959:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.820238:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641819904 00000100:00100000:0.0:1788845046.820248:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641819904 00002000:00100000:0.0:1788845046.820289:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2097151) req@ffff949f03a07100 x1875739641819904/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 520/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.820297:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a07100 with x1875739641819904 ext(1048576->2097151) 00002000:00010000:0.0:1788845046.820302:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a07100 00000100:00100000:0.0:1788845046.820328:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 613 00000100:00100000:0.0:1788845046.820335:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a07100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641819904:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845046.820994:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641820032 00000100:00100000:2.0:1788845046.821001:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641820032 00002000:00100000:2.0:1788845046.821030:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a06a00 x1875739641820032/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 504/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.821040:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a06a00 with x1875739641820032 00000100:00100000:2.0:1788845046.821169:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1034 00000100:00100000:2.0:1788845046.821179:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a06a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641820032:12345-192.168.201.29@tcp:3:fsx.0 00000004:00010000:2.0:1788845046.834262:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 712 00000100:00100000:2.0:1788845046.834509:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a06a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641820032:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 13325us (14544us total) trans 0 rc 954368/954368 00000100:00100000:2.0:1788845046.834521:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1034 00002000:00100000:2.0:1788845046.834536:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a06a00 x1875739641820032/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 504/440 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 954368/954368 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845046.834547:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a06a00 with x1875739641820032 00000100:00100000:0.0:1788845046.841770:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a07100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641819904:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 21423us (21901us total) trans 0 rc 991232/991232 00000100:00100000:0.0:1788845046.841787:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 613 00002000:00100000:0.0:1788845046.841802:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1048576->2097151) req@ffff949f03a07100 x1875739641819904/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:152/0 lens 520/440 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 991232/991232 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.841813:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a07100 with x1875739641819904 ext(1048576->2097151) 00002000:00010000:0.0:1788845046.841820:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a07100 00000100:00100000:3.0:1788845046.847168:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.848003:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641820160 00000100:00100000:1.0:1788845046.848014:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641820160 00000100:00100000:1.0:1788845046.848204:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1191 00000100:00100000:1.0:1788845046.848216:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:14891:x1875739641820160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845046.848236:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845046.848256:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa4fe expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.848290:0:7983:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.848299:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(13 17) 00010000:00010000:1.0:1788845046.848303:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 13) 00010000:00010000:1.0:1788845046.848309:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1867776->1925119] (req 1867776->1925119) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.848373:0:7983:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.848393:0:7983:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 5 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.848407:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660410368:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845046.848528:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.851375:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660410368:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845046.851417:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1867776->1925119] (req 1867776->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.851444:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:2.0:1788845046.851503:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641820160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3288us (4346us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845046.851512:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1191 00000100:00100000:3.0:1788845046.853099:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.853333:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641820288 00000100:00100000:0.0:1788845046.853343:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641820288 00010000:00010000:0.0:1788845046.853385:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 6 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.853420:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:0.0:1788845046.853427:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641820288:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845046.853439:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845046.853468:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 6 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.853489:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 6 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.853517:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000145c9bf0/0xb2187301287af04d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1437695) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4d4 expref: 6 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.853532:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 13) 00010000:00010000:0.0:1788845046.853537:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 13) 00010000:00010000:0.0:1788845046.853545:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.853553:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.853591:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 5 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.853616:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660410496:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845046.853714:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.860156:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.860300:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641820416 00000100:00100000:3.0:1788845046.860311:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641820416 00000100:00100000:3.0:1788845046.860384:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 614 00000100:00100000:3.0:1788845046.860395:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641820416:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:3.0:1788845046.860773:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:3.0:1788845046.860777:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00002000:00010000:3.0:1788845046.860779:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845046 -> 1788845046 00002000:00010000:3.0:1788845046.860782:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00002000:00010000:3.0:1788845046.860784:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 383 -> 383 00000100:00100000:3.0:1788845046.860857:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641820416:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 463us (712us total) trans 4294967511 rc 0/0 00000100:00100000:3.0:1788845046.860865:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 614 00000100:00100000:2.0:1788845046.863935:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.864279:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660410496:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845046.864303:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845046.864358:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641820288:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 10930us (11263us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.864368:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 552 00000100:00100000:1.0:1788845046.865564:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641820544 00000100:00100000:1.0:1788845046.865574:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641820544 00000100:00100000:1.0:1788845046.865635:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1192 00000100:00100000:1.0:1788845046.865645:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641820544:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845046.865662:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845046.865680:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b79c20b/0xb2187301287af07e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa505 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.865727:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660410624:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845046.865824:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845046.867866:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660410624:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845046.867938:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b79c20b/0xb2187301287af07e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa505 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.867965:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b79c20b/0xb2187301287af07e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa505 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.867981:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030400, rc 0) 00000100:00100000:3.0:1788845046.868035:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641820544:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2391us (4110us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845046.868045:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1192 00000100:00100000:1.0:1788845046.872322:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.872674:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641820672 00000100:00100000:0.0:1788845046.872684:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641820672 00000100:00100000:0.0:1788845046.872743:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1193 00000100:00100000:0.0:1788845046.872753:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641820672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.872771:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.872790:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287af085 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa50c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.872852:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660410752:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845046.872996:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845046.875951:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660410752:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845046.875994:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287af085 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa50c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.876021:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054d17b64/0xb2187301287af085 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa50c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.876034:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845046.876097:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641820672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3345us (3784us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845046.876105:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1193 00000100:00100000:1.0:1788845046.883599:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.883741:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641820928 00000100:00100000:2.0:1788845046.883752:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641820928 00000100:00100000:2.0:1788845046.883811:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1613 00000100:00100000:2.0:1788845046.883821:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641820928:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845046.883873:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845046.883932:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02031400/0xb2187301287aee4e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2dc expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.883950:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845046.883954:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845046.884254:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.884270:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.884279:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.884311:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.884330:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 9 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.884343:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660410880:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845046.884419:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845046.888767:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.889027:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660410880:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845046.889068:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.889621:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641821056 00000100:00100000:3.0:1788845046.889630:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641821056 00010000:00010000:3.0:1788845046.889670:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 10 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.889714:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 553 00000100:00100000:3.0:1788845046.889724:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641821056:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845046.889737:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845046.889744:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 10 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.889760:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 10 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.889782:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02030400/0xb2187301287aee78 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa2ff expref: 10 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.889797:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845046.889802:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.889811:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.889819:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.889846:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845046.889941:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641821056:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 218us (1183us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845046.889949:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 553 00010000:00010000:2.0:1788845046.890783:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.890793:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788845046.890927:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967864 00000004:00010000:2.0:1788845046.890970:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1011804 -> 1011804 00000004:00010000:2.0:1788845046.890973:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00000004:00010000:2.0:1788845046.890975:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:2.0:1788845046.890977:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00000004:00010000:2.0:1788845046.890978:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:2.0:1788845046.890984:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d88496f0 reply state 00000000c72bceac transno 4294967865 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.890989:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845046.891081:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641820928:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 7260us (7493us total) trans 4294967865 rc 0/0 00000100:00100000:2.0:1788845046.891088:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1613 00000020:00080000:2.0:1788845046.891091:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788845046.893460:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.893477:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.893543:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.893550:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845046.893555:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.893563:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.893583:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.893587:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845046.893591:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.893598:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af08c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.895420:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.895687:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641821184 00000100:00100000:2.0:1788845046.895697:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641821184 00000100:00100000:2.0:1788845046.895754:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1194 00000100:00100000:2.0:1788845046.895766:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641821184:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.895786:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.895804:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa513 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.895835:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.895845:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(1 13) 00010000:00010000:2.0:1788845046.895849:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:2.0:1788845046.895854:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.895931:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 6 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.895950:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 6 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.895964:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660411008:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845046.896136:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845046.900834:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.900922:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641821312 00000100:00100000:3.0:1788845046.900933:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641821312 00010000:00010000:3.0:1788845046.900974:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 7 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.901016:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 554 00000100:00100000:3.0:1788845046.901027:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641821312:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845046.901043:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845046.901337:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 7 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.901357:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 7 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.901383:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:3.0:1788845046.901388:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:3.0:1788845046.901396:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.901409:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.901449:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.901464:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660411136:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845046.901556:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845046.903778:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5826:1875739660411136:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845046.903798:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845046.903843:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2369:x1875739641821312:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2819us (3021us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845046.903852:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 554 00010000:00010000:2.0:1788845046.904776:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af077 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1867776->1925119) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4fe expref: 6 pid: 7983 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.904797:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660411008:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845046.904824:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.904841:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:2.0:1788845046.904934:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641821184:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9172us (9524us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845046.904940:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1194 00000100:00100000:3.0:1788845046.906997:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.907088:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.907239:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641821568 00000100:00100000:1.0:1788845046.907248:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641821568 00002000:00010000:1.0:1788845046.907281:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845046.907284:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a10380 with x1875739641821568 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845046.907294:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a10380. 00000100:00100000:1.0:1788845046.907321:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 615 00000100:00100000:1.0:1788845046.907329:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641821568:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845046.907843:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 2097152 -> 0 00002000:00010000:1.0:1788845046.907847:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00002000:00010000:1.0:1788845046.907848:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845046 -> 1788845046 00002000:00010000:1.0:1788845046.907850:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00002000:00010000:1.0:1788845046.907851:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 383 -> 383 00000100:00100000:1.0:1788845046.907927:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641821568:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 598us (837us total) trans 4294967512 rc 0/0 00000100:00100000:1.0:1788845046.907933:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 615 00002000:00010000:1.0:1788845046.907936:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:1.0:1788845046.907938:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a10380 with x1875739641821568 ext(0->18446744073709551615) 00002000:00010000:1.0:1788845046.907941:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a10380. 00000100:00100000:0.0:1788845046.911212:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641821440 00000100:00100000:0.0:1788845046.911227:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641821440 00002000:00010000:1.0:1788845046.911430:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845046.911436:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399b800 with x1875739641821440 00000100:00100000:1.0:1788845046.911818:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1035 00000100:00100000:1.0:1788845046.911829:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641821440:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845046.912789:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1011804 -> 13149 00000004:00010000:1.0:1788845046.912795:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00000004:00010000:1.0:1788845046.912797:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:1.0:1788845046.912799:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00000004:00010000:1.0:1788845046.912801:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845046.912945:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641821440:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 1116us (5952us total) trans 4294967866 rc 0/0 00000100:00100000:1.0:1788845046.912954:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1035 00002000:00010000:1.0:1788845046.912958:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845046.912961:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399b800 with x1875739641821440 00000100:00100000:0.0:1788845046.917994:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.918348:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641821696 00000100:00100000:2.0:1788845046.918360:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641821696 00000100:00100000:2.0:1788845046.918457:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1614 00000100:00100000:2.0:1788845046.918469:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641821696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.918504:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.918528:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af09a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa51a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.918628:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af0a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.918644:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af0a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.918657:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af0a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845046.918686:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 13149 00000004:00200000:2.0:1788845046.918690:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845046.918709:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af0a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.918739:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af09a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa51a expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.918759:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af0a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa51a expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.918769:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845046.918772:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845046.918776:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864200, rc 0) 00000100:00100000:2.0:1788845046.918841:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641821696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 375us (857us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845046.918848:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1614 00000100:00100000:0.0:1788845046.922344:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.922399:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.922420:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641821824 00000100:00100000:1.0:1788845046.922430:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641821824 00000100:00100000:1.0:1788845046.922617:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1615 00000100:00100000:1.0:1788845046.922629:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641821824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845046.922650:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845046.922670:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287af0a8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa521 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.922707:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.922748:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399f800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660411264:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845046.922831:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.922984:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641821952 00000100:00100000:2.0:1788845046.922990:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641821952 00000100:00100000:2.0:1788845046.923038:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1195 00000100:00100000:2.0:1788845046.923049:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641821952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845046.923059:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.923072:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af0af lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa528 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.923114:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660411392:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845046.923176:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845046.928779:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845046 00000100:00100000:2.0:1788845046.928815:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399f800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660411264:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845046.928840:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 13149 00010000:00010000:2.0:1788845046.928858:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287af0a8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa521 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.928879:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00000100:00100000:1.0:1788845046.929195:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660411392:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845046.929220:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af0af lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa528 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.929240:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af0af lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa528 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.929254:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865c00, rc 0) 00000100:00100000:1.0:1788845046.929304:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641821952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6258us (6904us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845046.929314:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1195 00010000:00010000:2.0:1788845046.929315:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845046.929325:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030e00/0xb2187301287af0a8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa521 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.929341:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030e00, rc 0) 00000100:00100000:2.0:1788845046.929380:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641821824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6754us (7045us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845046.929387:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1615 00000100:00100000:1.0:1788845046.934665:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.934711:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.934794:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641822080 00000100:00100000:3.0:1788845046.934801:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641822208 00000100:00100000:3.0:1788845046.934806:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641822208 00000100:00100000:2.0:1788845046.934806:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641822080 00000100:00100000:2.0:1788845046.934865:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1616 00000100:00100000:2.0:1788845046.934876:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641822080:12345-192.168.201.29@tcp:101:fsx.0 00000100:00100000:3.0:1788845046.934914:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1196 00000100:00100000:3.0:1788845046.934921:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641822208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845046.934931:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845046.934931:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845046.934947:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0b6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa536 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.934950:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af0bd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa52f expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.934999:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.935010:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a13800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660411520:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845046.935046:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660411648:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845046.935095:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.935096:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845046.936906:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a13800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660411520:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845046.936938:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0b6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa536 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.936956:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0b6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa536 expref: 7 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.936966:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:3.0:1788845046.937010:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641822208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2090us (2299us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845046.937016:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1196 00000100:00100000:2.0:1788845046.938827:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f20380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660411648:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845046.938850:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 13149 00010000:00010000:2.0:1788845046.938869:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af0bd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa52f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.938986:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845046.938989:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845046.938995:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af0bd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa52f expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.939008:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845046.939178:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641822080:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4303us (4708us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845046.939187:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1616 00000100:00100000:1.0:1788845046.942776:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.942873:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641822336 00000100:00100000:0.0:1788845046.942915:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641822336 00000100:00100000:0.0:1788845046.942972:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1617 00000100:00100000:0.0:1788845046.942984:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641822336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845046.943010:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845046.943031:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af0c4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa53d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.943143:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.943160:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.943171:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845046.943200:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 13149 00000004:00200000:0.0:1788845046.943205:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845046.943226:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.943254:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af0c4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.943271:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.943280:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.943283:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845046.943286:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007800, rc 0) 00000100:00100000:0.0:1788845046.943344:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641822336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 362us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.943351:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1617 00000100:00100000:2.0:1788845046.950575:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.950674:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641822592 00000100:00100000:1.0:1788845046.950683:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641822592 00000100:00100000:1.0:1788845046.950737:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1618 00000100:00100000:1.0:1788845046.950747:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641822592:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845046.950798:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845046.950825:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af0a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa51a expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.950844:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845046.950849:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845046.951082:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.951096:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.951111:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.951142:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.951160:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 17 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.951174:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399d180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660411776:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845046.951246:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845046.954700:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.954809:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399d180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660411776:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845046.954838:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.955578:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641822720 00000100:00100000:0.0:1788845046.955587:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641822720 00010000:00010000:2.0:1788845046.955768:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 18 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845046.955840:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 555 00000100:00100000:2.0:1788845046.955851:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a13800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641822720:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845046.955871:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845046.955879:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 18 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.956035:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 18 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.956135:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af0cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa53d expref: 18 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.956152:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845046.956157:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.956167:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.956175:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845046.956240:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845046.956315:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a13800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641822720:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 464us (1623us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845046.956323:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 555 00010000:00010000:1.0:1788845046.957515:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.957528:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845046.957660:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967867 00000004:00010000:1.0:1788845046.957721:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 13149 -> 13149 00000004:00010000:1.0:1788845046.957725:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00000004:00010000:1.0:1788845046.957728:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845046 -> 1788845043 00000004:00010000:1.0:1788845046.957731:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00000004:00010000:1.0:1788845046.957733:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000004:00010000:1.0:1788845046.957741:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000086f05e4 reply state 000000005f0a9f9d transno 4294967868 ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845046.957750:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845046.958166:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641822592:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 7415us (7597us total) trans 4294967868 rc 0/0 00000100:00100000:1.0:1788845046.958176:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1618 00000020:00080000:1.0:1788845046.958181:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845046.960100:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.960117:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.960134:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.960142:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a000031800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845046.960147:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.960156:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.960173:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845046.960177:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845046.960181:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845046.960188:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af0d2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.962542:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.962979:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641822848 00000100:00100000:3.0:1788845046.962989:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641822848 00000100:00100000:3.0:1788845046.963048:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1619 00000100:00100000:3.0:1788845046.963057:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641822848:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845046.963079:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845046.963098:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.963183:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.963195:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.963232:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.963250:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 16 pid: 5840 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.963264:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a11c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660411904:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845046.963334:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845046.966532:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.966732:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641822976 00000100:00100000:0.0:1788845046.966741:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641822976 00010000:00010000:0.0:1788845046.966783:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 17 pid: 5840 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.966823:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 556 00000100:00100000:0.0:1788845046.966832:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641822976:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845046.966847:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845046.966873:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 17 pid: 5840 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.966922:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 17 pid: 5840 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.966946:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845046.966952:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.966963:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.966973:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.967009:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.967023:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660412032:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845046.967103:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18600 going to sleep for 11 seconds 00010000:00010000:3.0:1788845046.969234:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031200/0xb2187301287af070 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa4f7 expref: 17 pid: 5840 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.969272:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a11c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660411904:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845046.969300:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.969321:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845046.969324:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845046.969327:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e800, rc 0) 00000100:00100000:3.0:1788845046.969596:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641822848:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6538us (7061us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845046.969614:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1619 00000100:00100000:0.0:1788845046.971057:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660412032:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845046.971079:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845046.971131:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641822976:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4291us (4601us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.971142:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 556 00000100:00100000:0.0:1788845046.975261:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845046.975396:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641823104 00000100:00100000:3.0:1788845046.975406:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641823104 00000100:00100000:3.0:1788845046.975642:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1197 00000100:00100000:3.0:1788845046.975654:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:14891:x1875739641823104:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845046.975675:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845046.975694:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa54b expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.975730:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.975739:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:3.0:1788845046.975744:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 13) 00010000:00010000:3.0:1788845046.975750:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.975787:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 6 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.975812:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 6 pid: 6682 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845046.975828:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a11c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660412160:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845046.975942:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845046.979372:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a11c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660412160:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845046.979412:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 7 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845046.979435:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:3.0:1788845046.979512:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:14891:x1875739641823104:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3859us (4259us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845046.979521:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1197 00000100:00100000:2.0:1788845046.979599:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.979753:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641823232 00000100:00100000:0.0:1788845046.979762:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641823232 00010000:00010000:0.0:1788845046.979793:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 7 pid: 6682 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.979831:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 557 00000100:00100000:0.0:1788845046.979841:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641823232:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845046.979855:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845046.979880:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 7 pid: 6682 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.979923:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 7 pid: 6682 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.979946:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af093 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa513 expref: 7 pid: 6682 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.979961:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 13) 00010000:00010000:0.0:1788845046.979966:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(3 13) 00010000:00010000:0.0:1788845046.979974:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.979987:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845046.980028:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845046.980053:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660412288:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845046.980113:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18600 going to sleep for 11 seconds 00000100:00100000:2.0:1788845046.984402:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845046.984440:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.984578:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641823360 00000100:00100000:1.0:1788845046.984589:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641823360 00002000:00010000:1.0:1788845046.984644:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845046.984649:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4a680 with x1875739641823360 00000100:00100000:1.0:1788845046.984691:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1036 00000100:00100000:1.0:1788845046.984703:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641823360:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845046.984964:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 13149 -> 281699 00000004:00010000:1.0:1788845046.984969:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00000004:00010000:1.0:1788845046.984971:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:1.0:1788845046.984973:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00000004:00010000:1.0:1788845046.984975:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:1.0:1788845046.985045:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641823360:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 344us (651us total) trans 4294967869 rc 0/0 00000100:00100000:1.0:1788845046.985053:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1036 00002000:00010000:1.0:1788845046.985057:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845046.985060:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4a680 with x1875739641823360 00000100:00100000:3.0:1788845046.985335:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641823488 00000100:00100000:3.0:1788845046.985341:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641823488 00002000:00010000:3.0:1788845046.985364:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845046.985368:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da49c00 with x1875739641823488 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845046.985375:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da49c00. 00000100:00100000:3.0:1788845046.985401:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 616 00000100:00100000:3.0:1788845046.985410:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641823488:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:3.0:1788845046.985699:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1788845046.985703:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845046 -> 1788845046 00002000:00010000:3.0:1788845046.985705:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845046 -> 1788845046 00002000:00010000:3.0:1788845046.985708:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845046 -> 1788845046 00002000:00010000:3.0:1788845046.985710:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 383 -> 383 00000100:00100000:3.0:1788845046.985738:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641823488:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 330us (1298us total) trans 4294967513 rc 0/0 00000100:00100000:3.0:1788845046.985746:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 616 00002000:00010000:3.0:1788845046.985750:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:3.0:1788845046.985752:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a02da49c00 with x1875739641823488 ext(0->18446744073709551615) 00002000:00010000:3.0:1788845046.985757:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a02da49c00. 00000100:00100000:0.0:1788845046.985943:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5824:1875739660412288:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845046.985970:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845046.985995:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641823232:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6156us (6397us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845046.986003:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 557 00000100:00100000:2.0:1788845046.989705:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845046.989781:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641823616 00000100:00100000:0.0:1788845046.989790:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641823616 00002000:00100000:0.0:1788845046.989838:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48a80 x1875739641823616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845046.989849:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48a80 with x1875739641823616 00000100:00100000:1.0:1788845046.989963:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1037 00000100:00100000:1.0:1788845046.989976:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641823616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.990233:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641823616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 260us (537us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.990241:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1037 00002000:00100000:1.0:1788845046.990249:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48a80 x1875739641823616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.990263:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48a80 with x1875739641823616 00000100:00100000:0.0:1788845046.993018:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.993287:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641823744 00000100:00100000:1.0:1788845046.993295:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641823744 00002000:00100000:1.0:1788845046.993336:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967800 x1875739641823744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.993345:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967800 with x1875739641823744 00000100:00100000:1.0:1788845046.993373:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1038 00000100:00100000:1.0:1788845046.993380:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641823744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.993755:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641823744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 373us (743us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.993763:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1038 00002000:00100000:1.0:1788845046.993770:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967800 x1875739641823744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.993779:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967800 with x1875739641823744 00000100:00100000:2.0:1788845046.997198:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845046.997346:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641823872 00000100:00100000:1.0:1788845046.997357:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641823872 00002000:00100000:1.0:1788845046.997406:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4aa00 x1875739641823872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/0 e 0 to 0 dl 1788845057 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.997416:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4aa00 with x1875739641823872 00000100:00100000:1.0:1788845046.997459:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1039 00000100:00100000:1.0:1788845046.997469:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641823872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845046.997759:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641823872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 290us (569us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845046.997771:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1039 00002000:00100000:1.0:1788845046.997778:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4aa00 x1875739641823872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:152/0 lens 488/4536 e 0 to 0 dl 1788845057 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845046.997785:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4aa00 with x1875739641823872 00000100:00100000:2.0:1788845047.001566:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.001686:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641824000 00000100:00100000:0.0:1788845047.001695:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641824000 00002000:00100000:0.0:1788845047.001761:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa300 x1875739641824000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.001771:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa300 with x1875739641824000 00000100:00100000:0.0:1788845047.001816:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1040 00000100:00100000:0.0:1788845047.001826:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641824000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.002117:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641824000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 292us (560us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.002125:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1040 00002000:00100000:0.0:1788845047.002133:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa300 x1875739641824000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.002140:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa300 with x1875739641824000 00000100:00080000:3.0:1788845047.002209:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845047.005856:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.006077:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641824128 00000100:00100000:1.0:1788845047.006088:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641824128 00002000:00100000:1.0:1788845047.006150:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15f80 x1875739641824128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.006161:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15f80 with x1875739641824128 00000100:00100000:1.0:1788845047.006190:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1041 00000100:00100000:1.0:1788845047.006201:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641824128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.006448:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641824128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (602us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.006456:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1041 00002000:00100000:1.0:1788845047.006463:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15f80 x1875739641824128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.006471:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15f80 with x1875739641824128 00000100:00100000:2.0:1788845047.009749:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.010346:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641824256 00000100:00100000:0.0:1788845047.010357:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641824256 00002000:00100000:0.0:1788845047.010408:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a14380 x1875739641824256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.010420:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a14380 with x1875739641824256 00000100:00100000:0.0:1788845047.010459:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1042 00000100:00100000:0.0:1788845047.010468:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641824256:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.010751:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641824256:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 282us (1009us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.010759:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1042 00002000:00100000:0.0:1788845047.010766:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a14380 x1875739641824256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.010774:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a14380 with x1875739641824256 00000100:00100000:2.0:1788845047.014979:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.015131:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641824384 00000100:00100000:1.0:1788845047.015143:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641824384 00002000:00100000:1.0:1788845047.015201:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a17800 x1875739641824384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.015213:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a17800 with x1875739641824384 00000100:00100000:1.0:1788845047.015249:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1043 00000100:00100000:1.0:1788845047.015258:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641824384:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.015670:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641824384:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 411us (697us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.015679:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1043 00002000:00100000:1.0:1788845047.015686:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a17800 x1875739641824384/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.015694:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a17800 with x1875739641824384 00000100:00100000:2.0:1788845047.018589:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.018739:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641824512 00000100:00100000:0.0:1788845047.018748:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641824512 00002000:00100000:0.0:1788845047.018799:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15c00 x1875739641824512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.018812:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15c00 with x1875739641824512 00000100:00100000:0.0:1788845047.018852:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1044 00000100:00100000:0.0:1788845047.018862:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641824512:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.019138:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641824512:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 276us (557us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.019147:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1044 00002000:00100000:0.0:1788845047.019155:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15c00 x1875739641824512/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.019163:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15c00 with x1875739641824512 00000100:00100000:2.0:1788845047.022817:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.022926:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641824640 00000100:00100000:0.0:1788845047.022937:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641824640 00002000:00100000:0.0:1788845047.022989:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a16300 x1875739641824640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.023002:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a16300 with x1875739641824640 00000100:00100000:0.0:1788845047.023044:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1045 00000100:00100000:0.0:1788845047.023056:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641824640:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.023339:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641824640:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 284us (531us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.023346:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1045 00002000:00100000:0.0:1788845047.023353:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a16300 x1875739641824640/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.023360:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a16300 with x1875739641824640 00000100:00100000:2.0:1788845047.026753:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.027156:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641824768 00000100:00100000:0.0:1788845047.027166:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641824768 00002000:00100000:0.0:1788845047.027215:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15500 x1875739641824768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.027226:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15500 with x1875739641824768 00000100:00100000:0.0:1788845047.027267:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1046 00000100:00100000:0.0:1788845047.027278:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641824768:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.027568:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641824768:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 291us (824us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.027576:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1046 00002000:00100000:0.0:1788845047.027583:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15500 x1875739641824768/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.027591:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15500 with x1875739641824768 00000100:00100000:2.0:1788845047.031074:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.031177:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641824896 00000100:00100000:0.0:1788845047.031188:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641824896 00002000:00100000:0.0:1788845047.031234:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15180 x1875739641824896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.031243:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15180 with x1875739641824896 00000100:00100000:0.0:1788845047.031274:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1047 00000100:00100000:0.0:1788845047.031281:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641824896:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.031583:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641824896:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 301us (521us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.031589:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1047 00002000:00100000:0.0:1788845047.031594:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15180 x1875739641824896/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.031599:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15180 with x1875739641824896 00000100:00100000:2.0:1788845047.034256:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.034449:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641825024 00000100:00100000:0.0:1788845047.034459:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641825024 00002000:00100000:0.0:1788845047.034514:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a17480 x1875739641825024/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.034523:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a17480 with x1875739641825024 00000100:00100000:0.0:1788845047.034609:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1048 00000100:00100000:0.0:1788845047.034620:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2367:x1875739641825024:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845047.036781:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 281699 -> 281699 00000004:00010000:3.0:1788845047.036790:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845046 -> 1788845047 00000004:00010000:3.0:1788845047.036792:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845047.036793:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845046 -> 1788845047 00000004:00010000:3.0:1788845047.036795:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 712 00000100:00100000:3.0:1788845047.036856:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641825024:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 2237us (2606us total) trans 4294967870 rc 0/0 00000100:00100000:3.0:1788845047.036864:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1048 00002000:00100000:3.0:1788845047.036872:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a17480 x1875739641825024/t4294967870(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/448 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.036878:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a17480 with x1875739641825024 00000100:00100000:2.0:1788845047.038979:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.039068:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641825152 00000100:00100000:1.0:1788845047.039075:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641825152 00000100:00100000:1.0:1788845047.039157:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1620 00000100:00100000:1.0:1788845047.039168:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a16680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641825152:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:2.0:1788845047.065850:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967869 is committed 00000001:00080000:2.0:1788845047.065869:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967866 is committed 00000001:00080000:2.0:1788845047.065877:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967870 is committed 00000100:00100000:3.0:1788845047.066079:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a16680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641825152:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 26907us (27100us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.066092:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1620 00000100:00100000:2.0:1788845047.068548:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.068638:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641825280 00000100:00100000:0.0:1788845047.068648:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641825280 00000100:00100000:0.0:1788845047.068728:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1621 00000100:00100000:0.0:1788845047.068740:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641825280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.068765:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.068787:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af0e7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa552 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.068865:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.068876:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.068914:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845047.068943:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 281699 00000004:00200000:0.0:1788845047.068947:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845047.068966:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.068987:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af0e7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.069000:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.069010:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845047.069013:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845047.069015:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006c00, rc 0) 00000100:00100000:0.0:1788845047.069061:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641825280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 324us (524us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845047.069068:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1621 00000100:00080000:3.0:1788845047.069137:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845047.074410:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.074571:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641825408 00000100:00100000:2.0:1788845047.074582:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641825408 00000100:00100000:2.0:1788845047.074693:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1622 00000100:00100000:2.0:1788845047.074704:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a11c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641825408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.074725:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.074746:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af0f5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa559 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.074783:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.074828:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a14700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660412416:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.074930:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.076097:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.076149:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641825536 00000100:00100000:1.0:1788845047.076157:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641825536 00000100:00100000:1.0:1788845047.076229:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1198 00000100:00100000:1.0:1788845047.076240:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641825536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.076255:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.076269:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af0fc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa560 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.076320:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660412544:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845047.076385:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00080000:3.0:1788845047.076515:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00010000:2.0:1788845047.078345:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845047 00000100:00100000:2.0:1788845047.078381:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a14700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660412416:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845047.078649:0:6682:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845046 -> 1788845047 00002000:00010000:1.0:1788845047.078654:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845046 -> 1788845047 00002000:00010000:1.0:1788845047.078656:0:6682:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845046 -> 1788845047 00000100:00100000:1.0:1788845047.078670:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660412544:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845047.078698:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af0fc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa560 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.078720:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4425405/0xb2187301287af0fc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa560 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.078732:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:1.0:1788845047.078792:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641825536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2553us (2700us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845047.078800:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1198 00002000:00010000:2.0:1788845047.078926:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 281699 00010000:00010000:2.0:1788845047.078940:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af0f5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa559 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.078954:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.078956:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845047.078961:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af0f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa559 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.078972:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845047.078996:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a11c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641825408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4294us (4596us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.079002:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1622 00000100:00100000:0.0:1788845047.083365:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.083471:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.083950:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641825792 00000100:00100000:2.0:1788845047.083959:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641825792 00000100:00100000:2.0:1788845047.084018:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1199 00000100:00100000:2.0:1788845047.084029:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aaa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641825792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.084049:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.084067:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af103 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa56e expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.084138:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660412672:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.084211:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845047.084225:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641825664 00000100:00100000:1.0:1788845047.084229:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641825664 00000100:00100000:1.0:1788845047.084271:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1623 00000100:00100000:1.0:1788845047.084280:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641825664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.084290:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.084303:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af10a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa567 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.084334:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 9 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.084360:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399f480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660412800:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845047.084387:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788845047.088095:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399f480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660412800:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845047.088134:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 281699 00010000:00010000:1.0:1788845047.088153:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af10a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa567 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.088182:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845047.088185:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845047.088190:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000031800/0xb2187301287af10a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa567 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.088207:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031800, rc 0) 00000100:00100000:2.0:1788845047.088263:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660412672:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.088286:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af103 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa56e expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.088310:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af103 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa56e expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.088319:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845047.088344:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aaa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641825792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4317us (4873us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.088353:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1199 00000100:00100000:1.0:1788845047.089129:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641825664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4839us (5762us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845047.089138:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1623 00000100:00100000:0.0:1788845047.091585:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.091671:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641826048 00000100:00100000:3.0:1788845047.091679:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641826048 00000100:00100000:3.0:1788845047.091730:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1624 00000100:00100000:3.0:1788845047.091738:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641826048:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:3.0:1788845047.092048:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.092066:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.092075:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.092106:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.092124:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 13 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.092153:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660412928:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845047.092233:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.095348:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.095461:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660412928:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845047.095498:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.095672:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641826176 00000100:00100000:1.0:1788845047.095679:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641826176 00010000:00010000:1.0:1788845047.095712:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 14 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.095753:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 558 00000100:00100000:1.0:1788845047.095763:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d12850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641826176:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845047.095776:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845047.095783:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 14 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.095800:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 14 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.095823:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af0ee lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa552 expref: 14 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.095844:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845047.095850:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.095860:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.095868:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.095928:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845047.095974:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d12850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641826176:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 212us (636us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845047.095982:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 558 00010000:00010000:3.0:1788845047.096069:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.096077:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845047.096197:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967871 00000004:00010000:3.0:1788845047.096245:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 281699 -> 281699 00000004:00010000:3.0:1788845047.096249:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00000004:00010000:3.0:1788845047.096251:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845047 -> 1788845043 00000004:00010000:3.0:1788845047.096253:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00000004:00010000:3.0:1788845047.096255:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 712 -> 198 00000004:00010000:3.0:1788845047.096262:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000bcc0b74 reply state 00000000635838a4 transno 4294967872 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.096270:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845047.096348:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641826048:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4609us (4775us total) trans 4294967872 rc 0/0 00000100:00100000:3.0:1788845047.096355:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1624 00000020:00080000:3.0:1788845047.096359:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845047.098545:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.098561:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.098568:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.098574:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216fe00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845047.098579:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.098587:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.098614:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.098617:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845047.098622:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.098633:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af111 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.100622:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.100816:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641826304 00000100:00100000:1.0:1788845047.100826:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641826304 00002000:00010000:1.0:1788845047.100875:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1291195->18446744073709551615) 00002000:00100000:1.0:1788845047.100879:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a14700 with x1875739641826304 ext(1291195->18446744073709551615) 00002000:00010000:1.0:1788845047.100916:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a14700. 00000100:00100000:1.0:1788845047.100944:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 617 00000100:00100000:1.0:1788845047.100953:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641826304:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:1.0:1788845047.101165:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 1291195 00002000:00010000:1.0:1788845047.101170:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00002000:00010000:1.0:1788845047.101172:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845047 -> 1788845046 00002000:00010000:1.0:1788845047.101175:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00002000:00010000:1.0:1788845047.101177:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 383 -> 383 00000100:00100000:1.0:1788845047.101244:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641826304:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 290us (653us total) trans 4294967514 rc 0/0 00000100:00100000:1.0:1788845047.101251:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 617 00002000:00010000:1.0:1788845047.101262:0:6688:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1291195->18446744073709551615) 00002000:00100000:1.0:1788845047.101265:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a14700 with x1875739641826304 ext(1291195->18446744073709551615) 00002000:00010000:1.0:1788845047.101270:0:6688:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a14700. 00000100:00080000:3.0:1788845047.102240:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845047.107542:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.107990:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641826432 00000100:00100000:1.0:1788845047.108001:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641826432 00002000:00100000:1.0:1788845047.108049:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1228800->1232895) req@ffff949f03aa8700 x1875739641826432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.108060:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa8700 with x1875739641826432 ext(1228800->1232895) 00002000:00010000:1.0:1788845047.108066:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa8700 00000100:00100000:1.0:1788845047.108146:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 618 00000100:00100000:1.0:1788845047.108158:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641826432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.108660:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641826432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 316us (941us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.108671:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 618 00002000:00100000:1.0:1788845047.108679:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1228800->1232895) req@ffff949f03aa8700 x1875739641826432/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.108687:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa8700 with x1875739641826432 ext(1228800->1232895) 00002000:00010000:1.0:1788845047.108691:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa8700 00000100:00100000:0.0:1788845047.112697:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.112801:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641826560 00000100:00100000:1.0:1788845047.112811:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641826560 00002000:00100000:1.0:1788845047.112876:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1232896->1236991) req@ffff949f03aaad80 x1875739641826560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.112928:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aaad80 with x1875739641826560 ext(1232896->1236991) 00002000:00010000:1.0:1788845047.112935:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aaad80 00000100:00100000:1.0:1788845047.112963:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 619 00000100:00100000:1.0:1788845047.112972:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aaad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641826560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.113210:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aaad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641826560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 237us (523us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.113218:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 619 00002000:00100000:1.0:1788845047.113225:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1232896->1236991) req@ffff949f03aaad80 x1875739641826560/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.113233:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aaad80 with x1875739641826560 ext(1232896->1236991) 00002000:00010000:1.0:1788845047.113237:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aaad80 00000100:00100000:2.0:1788845047.117801:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.117980:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641826688 00000100:00100000:3.0:1788845047.117990:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641826688 00002000:00100000:2.0:1788845047.118199:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1236992->1241087) req@ffff949f03a16d80 x1875739641826688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.118211:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a16d80 with x1875739641826688 ext(1236992->1241087) 00002000:00010000:2.0:1788845047.118217:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a16d80 00000100:00100000:2.0:1788845047.118289:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 620 00000100:00100000:2.0:1788845047.118298:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641826688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.118678:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641826688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 379us (884us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.118686:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 620 00002000:00100000:2.0:1788845047.118694:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1236992->1241087) req@ffff949f03a16d80 x1875739641826688/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.118703:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a16d80 with x1875739641826688 ext(1236992->1241087) 00002000:00010000:2.0:1788845047.118707:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a16d80 00000100:00100000:0.0:1788845047.121729:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.121829:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641826816 00000100:00100000:1.0:1788845047.121839:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641826816 00002000:00100000:1.0:1788845047.121922:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1241088->1245183) req@ffff949f03aa8e00 x1875739641826816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.121933:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa8e00 with x1875739641826816 ext(1241088->1245183) 00002000:00010000:1.0:1788845047.121938:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa8e00 00000100:00100000:1.0:1788845047.121967:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 621 00000100:00100000:1.0:1788845047.121976:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641826816:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.122207:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641826816:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (485us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.122215:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 621 00002000:00100000:1.0:1788845047.122223:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1241088->1245183) req@ffff949f03aa8e00 x1875739641826816/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.122231:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa8e00 with x1875739641826816 ext(1241088->1245183) 00002000:00010000:1.0:1788845047.122235:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa8e00 00000100:00100000:0.0:1788845047.126129:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.126219:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641826944 00000100:00100000:1.0:1788845047.126229:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641826944 00002000:00100000:1.0:1788845047.126274:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff949f03aa8a80 x1875739641826944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.126285:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa8a80 with x1875739641826944 ext(1245184->1249279) 00002000:00010000:1.0:1788845047.126290:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa8a80 00000100:00100000:1.0:1788845047.126368:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 622 00000100:00100000:1.0:1788845047.126377:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641826944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.126594:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641826944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 217us (473us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.126612:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 622 00002000:00100000:1.0:1788845047.126619:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1245184->1249279) req@ffff949f03aa8a80 x1875739641826944/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.126628:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa8a80 with x1875739641826944 ext(1245184->1249279) 00002000:00010000:1.0:1788845047.126631:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa8a80 00000100:00100000:0.0:1788845047.129832:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.130130:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641827072 00000100:00100000:2.0:1788845047.130141:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641827072 00002000:00100000:2.0:1788845047.130190:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1253375) req@ffff949f03aa8000 x1875739641827072/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.130200:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa8000 with x1875739641827072 ext(1249280->1253375) 00002000:00010000:2.0:1788845047.130205:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa8000 00000100:00100000:2.0:1788845047.130238:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 623 00000100:00100000:2.0:1788845047.130247:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641827072:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.130637:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641827072:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 389us (812us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.130644:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 623 00002000:00100000:2.0:1788845047.130651:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1249280->1253375) req@ffff949f03aa8000 x1875739641827072/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.130659:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa8000 with x1875739641827072 ext(1249280->1253375) 00002000:00010000:2.0:1788845047.130662:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa8000 00000100:00100000:2.0:1788845047.134046:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.134116:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641827200 00000100:00100000:3.0:1788845047.134135:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641827200 00002000:00100000:3.0:1788845047.134187:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1257471) req@ffff949f03a14e00 x1875739641827200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.134199:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a14e00 with x1875739641827200 ext(1253376->1257471) 00002000:00010000:3.0:1788845047.134204:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a14e00 00000100:00100000:3.0:1788845047.134244:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 624 00000100:00100000:3.0:1788845047.134255:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641827200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.134823:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641827200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 566us (783us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.134832:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 624 00002000:00100000:3.0:1788845047.134840:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1257471) req@ffff949f03a14e00 x1875739641827200/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.134849:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a14e00 with x1875739641827200 ext(1253376->1257471) 00002000:00010000:3.0:1788845047.134852:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a14e00 00000100:00100000:2.0:1788845047.138331:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.138467:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641827328 00000100:00100000:3.0:1788845047.138534:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641827328 00002000:00100000:3.0:1788845047.138589:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1257472->1261567) req@ffff949f03a14a80 x1875739641827328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.138612:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a14a80 with x1875739641827328 ext(1257472->1261567) 00002000:00010000:3.0:1788845047.138618:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a14a80 00000100:00100000:3.0:1788845047.138656:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 625 00000100:00100000:3.0:1788845047.138666:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641827328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.139029:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641827328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 217us (561us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.139039:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 625 00002000:00100000:3.0:1788845047.139047:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1257472->1261567) req@ffff949f03a14a80 x1875739641827328/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.139057:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a14a80 with x1875739641827328 ext(1257472->1261567) 00002000:00010000:3.0:1788845047.139061:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a14a80 00000100:00100000:3.0:1788845047.142686:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.142848:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641827456 00000100:00100000:1.0:1788845047.142859:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641827456 00002000:00100000:1.0:1788845047.142943:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1261568->1265663) req@ffff949f03a12300 x1875739641827456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.142955:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a12300 with x1875739641827456 ext(1261568->1265663) 00002000:00010000:1.0:1788845047.142961:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a12300 00000100:00100000:1.0:1788845047.142991:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 626 00000100:00100000:1.0:1788845047.143001:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641827456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.143278:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641827456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 276us (601us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.143286:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 626 00002000:00100000:1.0:1788845047.143293:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1261568->1265663) req@ffff949f03a12300 x1875739641827456/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.143302:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a12300 with x1875739641827456 ext(1261568->1265663) 00002000:00010000:1.0:1788845047.143306:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a12300 00000100:00100000:1.0:1788845047.146470:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.146666:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641827584 00000100:00100000:2.0:1788845047.146677:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641827584 00002000:00100000:2.0:1788845047.146722:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1265664->1269759) req@ffff949f0399f100 x1875739641827584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.146732:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399f100 with x1875739641827584 ext(1265664->1269759) 00002000:00010000:2.0:1788845047.146736:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399f100 00000100:00100000:2.0:1788845047.146766:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 627 00000100:00100000:2.0:1788845047.146775:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641827584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.147018:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641827584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (555us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.147027:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 627 00002000:00100000:2.0:1788845047.147035:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1265664->1269759) req@ffff949f0399f100 x1875739641827584/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.147049:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399f100 with x1875739641827584 ext(1265664->1269759) 00002000:00010000:2.0:1788845047.147053:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399f100 00000100:00100000:2.0:1788845047.150613:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.151005:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641827712 00000100:00100000:3.0:1788845047.151017:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641827712 00002000:00100000:2.0:1788845047.151205:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1269760->1273855) req@ffff949f03a14000 x1875739641827712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.151217:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a14000 with x1875739641827712 ext(1269760->1273855) 00002000:00010000:2.0:1788845047.151222:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a14000 00000100:00100000:2.0:1788845047.151287:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 628 00000100:00100000:2.0:1788845047.151297:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641827712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.151572:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641827712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 276us (977us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.151579:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 628 00002000:00100000:2.0:1788845047.151586:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1269760->1273855) req@ffff949f03a14000 x1875739641827712/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.151594:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a14000 with x1875739641827712 ext(1269760->1273855) 00002000:00010000:2.0:1788845047.151597:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a14000 00000100:00100000:3.0:1788845047.155298:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.155583:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641827840 00000100:00100000:2.0:1788845047.155593:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641827840 00002000:00100000:2.0:1788845047.155648:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff949f03a11500 x1875739641827840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.155658:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a11500 with x1875739641827840 ext(1273856->1277951) 00002000:00010000:2.0:1788845047.155664:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a11500 00000100:00100000:2.0:1788845047.155699:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 629 00000100:00100000:2.0:1788845047.155708:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641827840:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.155980:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641827840:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 270us (690us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.155988:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 629 00002000:00100000:2.0:1788845047.155995:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1273856->1277951) req@ffff949f03a11500 x1875739641827840/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.156003:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a11500 with x1875739641827840 ext(1273856->1277951) 00002000:00010000:2.0:1788845047.156007:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a11500 00000100:00100000:1.0:1788845047.159556:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.159799:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641827968 00000100:00100000:0.0:1788845047.159809:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641827968 00002000:00100000:0.0:1788845047.159858:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1277952->1282047) req@ffff949f0399c380 x1875739641827968/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.159869:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399c380 with x1875739641827968 ext(1277952->1282047) 00002000:00010000:0.0:1788845047.159875:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399c380 00000100:00100000:0.0:1788845047.159942:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 630 00000100:00100000:0.0:1788845047.159955:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641827968:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.160224:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641827968:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 271us (675us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.160232:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 630 00002000:00100000:0.0:1788845047.160239:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1277952->1282047) req@ffff949f0399c380 x1875739641827968/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.160247:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399c380 with x1875739641827968 ext(1277952->1282047) 00002000:00010000:0.0:1788845047.160250:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399c380 00000100:00100000:0.0:1788845047.163672:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.163772:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641828096 00000100:00100000:2.0:1788845047.163781:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641828096 00002000:00100000:2.0:1788845047.163840:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1282048->1286143) req@ffff949f03aaaa00 x1875739641828096/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.163851:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aaaa00 with x1875739641828096 ext(1282048->1286143) 00002000:00010000:2.0:1788845047.163856:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aaaa00 00000100:00100000:2.0:1788845047.163917:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 631 00000100:00100000:2.0:1788845047.163927:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aaaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641828096:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.164149:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aaaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2368:x1875739641828096:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 222us (486us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.164156:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 631 00002000:00100000:2.0:1788845047.164163:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1282048->1286143) req@ffff949f03aaaa00 x1875739641828096/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.164171:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aaaa00 with x1875739641828096 ext(1282048->1286143) 00002000:00010000:2.0:1788845047.164175:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aaaa00 00000100:00100000:3.0:1788845047.168257:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.168412:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641828224 00000100:00100000:0.0:1788845047.168424:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641828224 00002000:00100000:0.0:1788845047.168488:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1286144->1290239) req@ffff949f03a11180 x1875739641828224/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.168500:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a11180 with x1875739641828224 ext(1286144->1290239) 00002000:00010000:0.0:1788845047.168506:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a11180 00000100:00100000:0.0:1788845047.168544:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 632 00000100:00100000:0.0:1788845047.168554:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a11180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641828224:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.168812:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a11180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641828224:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (563us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.168820:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 632 00002000:00100000:0.0:1788845047.168827:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1286144->1290239) req@ffff949f03a11180 x1875739641828224/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.168835:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a11180 with x1875739641828224 ext(1286144->1290239) 00002000:00010000:0.0:1788845047.168838:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a11180 00000100:00100000:0.0:1788845047.173095:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.173313:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641828352 00000100:00100000:2.0:1788845047.173325:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641828352 00002000:00100000:2.0:1788845047.173374:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1290240->1294335) req@ffff949f03aa9880 x1875739641828352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.173385:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa9880 with x1875739641828352 ext(1290240->1294335) 00002000:00010000:2.0:1788845047.173391:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa9880 00000100:00100000:2.0:1788845047.173428:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 633 00000100:00100000:2.0:1788845047.173438:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641828352:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.173708:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641828352:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 269us (622us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.173715:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 633 00002000:00100000:2.0:1788845047.173722:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1290240->1294335) req@ffff949f03aa9880 x1875739641828352/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.173729:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aa9880 with x1875739641828352 ext(1290240->1294335) 00002000:00010000:2.0:1788845047.173733:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aa9880 00000100:00100000:1.0:1788845047.178184:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.178360:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641828480 00000100:00100000:0.0:1788845047.178371:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641828480 00002000:00100000:0.0:1788845047.178420:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1228800->1291194) req@ffff949f0399d180 x1875739641828480/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.178431:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399d180 with x1875739641828480 ext(1228800->1291194) 00002000:00010000:0.0:1788845047.178436:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399d180 00000100:00100000:0.0:1788845047.178475:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 634 00000100:00100000:0.0:1788845047.178495:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641828480:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845047.182074:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641828480:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3570us (3892us total) trans 4294967515 rc 0/0 00000100:00100000:0.0:1788845047.182088:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 634 00002000:00100000:0.0:1788845047.182101:0:6688:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (1228800->1291194) req@ffff949f0399d180 x1875739641828480/t4294967515(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/448 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.182128:0:6688:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399d180 with x1875739641828480 ext(1228800->1291194) 00002000:00010000:0.0:1788845047.182135:0:6688:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399d180 00000100:00100000:3.0:1788845047.187591:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.187702:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641828608 00000100:00100000:2.0:1788845047.187713:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641828608 00000100:00100000:2.0:1788845047.187776:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1200 00000100:00100000:2.0:1788845047.187786:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641828608:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845047.239918:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967515 is committed 00000001:00080000:1.0:1788845047.239952:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967512 is committed 00000100:00100000:3.0:1788845047.240267:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2369:x1875739641828608:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 52477us (52682us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.240279:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1200 00000100:00100000:1.0:1788845047.243485:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.243549:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641828736 00000100:00100000:2.0:1788845047.243559:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641828736 00000100:00100000:2.0:1788845047.243648:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1625 00000100:00100000:2.0:1788845047.243659:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641828736:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.243678:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.243699:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af118 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa575 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.243781:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.243797:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.243808:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845047.243834:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1291195 00000004:00200000:2.0:1788845047.243838:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845047.243856:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.243920:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af118 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.243940:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.243949:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.243952:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845047.243955:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864e00, rc 0) 00000100:00100000:2.0:1788845047.244012:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641828736:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 355us (537us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845047.244019:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1625 00000100:00100000:1.0:1788845047.248413:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.248503:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.248635:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641828992 00000100:00100000:2.0:1788845047.248635:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641828864 00000100:00100000:3.0:1788845047.248645:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641828992 00000100:00100000:2.0:1788845047.248645:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641828864 00000100:00100000:3.0:1788845047.248693:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1201 00000100:00100000:2.0:1788845047.248696:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1626 00000100:00100000:3.0:1788845047.248703:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641828992:12345-192.168.201.29@tcp:101:fsx.0 00000100:00100000:2.0:1788845047.248703:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641828864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.248721:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.248721:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.248736:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287af12d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa583 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.248737:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af126 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa57c expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.248767:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.248787:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660413056:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.248787:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a16a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660413184:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.249126:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.256833:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845047.259969:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845047 00000100:00100000:2.0:1788845047.260002:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a16a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660413184:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845047.260025:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 281699 00010000:00010000:2.0:1788845047.260046:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af126 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa57c expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.260067:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.260071:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845047.260076:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af126 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa57c expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.260093:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864400, rc 0) 00000100:00100000:2.0:1788845047.260152:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641828864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11450us (11648us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.260162:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1626 00002000:00010000:3.0:1788845047.261613:0:6685:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845046 -> 1788845047 00000100:00100000:0.0:1788845047.261746:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660413056:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.261781:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287af12d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa583 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.261803:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287af12d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa583 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.261816:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:0.0:1788845047.261931:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641828992:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 13228us (13526us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.261943:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1201 00000100:00100000:1.0:1788845047.265061:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.265236:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641829248 00000100:00100000:2.0:1788845047.265246:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641829248 00000100:00100000:2.0:1788845047.265309:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1202 00000100:00100000:2.0:1788845047.265320:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641829248:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.265339:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.265356:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af134 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa591 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.265414:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a16a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660413312:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.265585:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845047.265680:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.266241:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641829120 00000100:00100000:3.0:1788845047.266248:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641829120 00000100:00100000:3.0:1788845047.266291:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1627 00000100:00100000:3.0:1788845047.266300:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641829120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.266314:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.266327:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af13b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa58a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.266359:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.266394:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a13b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660413440:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.266905:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.271418:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a13b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660413440:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845047.271442:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 281699 00010000:00010000:3.0:1788845047.271461:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af13b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa58a expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.271491:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.271495:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.271500:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af13b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa58a expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.271516:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845047.271576:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641829120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5277us (5896us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.271584:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1627 00000100:00100000:2.0:1788845047.273053:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a16a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660413312:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.273086:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af134 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa591 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.273213:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af134 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa591 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.273226:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:2.0:1788845047.273379:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641829248:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8060us (8326us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.273389:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1202 00000100:00100000:1.0:1788845047.284804:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.284848:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.285097:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641829504 00000100:00100000:3.0:1788845047.285118:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641829504 00002000:00100000:3.0:1788845047.285169:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1228799) req@ffff949f0399f800 x1875739641829504/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.285182:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399f800 with x1875739641829504 ext(1048576->1228799) 00002000:00010000:3.0:1788845047.285188:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399f800 00000100:00100000:3.0:1788845047.285212:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 635 00000100:00100000:3.0:1788845047.285221:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641829504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.286507:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641829376 00000100:00100000:0.0:1788845047.286518:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641829376 00002000:00100000:0.0:1788845047.286555:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399ce00 x1875739641829376/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 504/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.286566:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399ce00 with x1875739641829376 00000100:00100000:0.0:1788845047.286608:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1049 00000100:00100000:0.0:1788845047.286616:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641829376:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.291772:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2370:x1875739641829504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 6540us (6911us total) trans 0 rc 180224/180224 00000100:00100000:3.0:1788845047.291787:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 635 00002000:00100000:3.0:1788845047.291800:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1228799) req@ffff949f0399f800 x1875739641829504/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 180224/180224 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.291812:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f0399f800 with x1875739641829504 ext(1048576->1228799) 00002000:00010000:3.0:1788845047.291819:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f0399f800 00000100:00100000:0.0:1788845047.294578:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641829376:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 7951us (9771us total) trans 0 rc 237568/237568 00000100:00100000:0.0:1788845047.294593:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1049 00002000:00100000:0.0:1788845047.294618:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399ce00 x1875739641829376/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 504/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 237568/237568 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.294629:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399ce00 with x1875739641829376 00000100:00100000:1.0:1788845047.300099:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.300367:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.300449:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641829632 00000100:00100000:3.0:1788845047.300459:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641829632 00000100:00100000:3.0:1788845047.300531:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1628 00000100:00100000:3.0:1788845047.300542:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641829632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.300563:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.300583:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af142 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa598 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.300632:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.300676:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a13b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660413568:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.300941:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.301026:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641829760 00000100:00100000:2.0:1788845047.301032:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641829760 00000100:00100000:2.0:1788845047.301127:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1203 00000100:00100000:2.0:1788845047.301137:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641829760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.301147:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.301160:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af149 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa59f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.301200:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a16a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660413696:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.301230:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.304179:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a13b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660413568:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845047.304207:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 281699 00010000:00010000:3.0:1788845047.304226:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af142 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa598 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.304253:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.304266:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.304272:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af142 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa598 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.304288:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845047.304359:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641829632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3819us (3992us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.304367:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1628 00000100:00100000:0.0:1788845047.305974:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a16a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660413696:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.306008:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af149 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa59f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.306031:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af149 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa59f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.306044:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:0.0:1788845047.306101:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641829760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4966us (6014us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.306110:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1203 00000100:00100000:3.0:1788845047.309195:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.309246:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.309464:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641829888 00000100:00100000:2.0:1788845047.309474:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641829888 00000100:00100000:2.0:1788845047.309537:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1629 00000100:00100000:2.0:1788845047.309546:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a13b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641829888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.309568:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.309588:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af150 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5a6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.309639:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.309679:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660413824:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.309750:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845047.312040:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641830016 00000100:00100000:1.0:1788845047.312053:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641830016 00000100:00100000:1.0:1788845047.312128:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1204 00000100:00100000:1.0:1788845047.312138:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641830016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.312156:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.312172:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000035601989/0xb2187301287af157 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5ad expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.312224:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03998380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660413952:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845047.312312:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.314159:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660413824:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845047.314185:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 281699 00010000:00010000:0.0:1788845047.314205:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af150 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5a6 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.314228:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845047.314231:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845047.314237:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af150 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5a6 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.314253:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865e00, rc 0) 00000100:00100000:0.0:1788845047.314655:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a13b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641829888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5108us (5467us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.314664:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1629 00000100:00100000:2.0:1788845047.316792:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03998380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660413952:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.316832:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000035601989/0xb2187301287af157 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5ad expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.316857:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000035601989/0xb2187301287af157 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5ad expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.316869:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000031200, rc 0) 00000100:00100000:2.0:1788845047.316933:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641830016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4796us (7686us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.316941:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1204 00000100:00100000:0.0:1788845047.321092:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.321347:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641830144 00000100:00100000:1.0:1788845047.321359:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641830144 00000100:00100000:1.0:1788845047.321459:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1630 00000100:00100000:1.0:1788845047.321481:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641830144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.321501:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.321521:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287af15e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5b4 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.321623:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287af165 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.321639:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287af165 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.321652:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287af165 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845047.321681:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1291195 00000004:00200000:1.0:1788845047.321685:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845047.321705:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287af165 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.321735:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030400/0xb2187301287af15e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5b4 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.321752:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287af165 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5b4 expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.321760:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845047.321762:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845047.321765:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a000030a00, rc 0) 00000100:00100000:1.0:1788845047.321870:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641830144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 401us (786us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845047.321877:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1630 00000100:00100000:2.0:1788845047.326552:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.326637:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.326711:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641830272 00000100:00100000:0.0:1788845047.326720:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641830272 00000100:00100000:0.0:1788845047.326784:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1631 00000100:00100000:0.0:1788845047.326794:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641830272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.326813:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.326832:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287af16c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5bb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.326868:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.326944:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660414080:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.327020:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.328616:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641830400 00000100:00100000:3.0:1788845047.328628:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641830400 00000100:00100000:0.0:1788845047.330036:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660414080:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845047.330063:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 281699 00010000:00010000:0.0:1788845047.330082:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287af16c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5bb expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.330111:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845047.330121:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845047.330127:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287af16c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5bb expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.330144:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.330206:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641830272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3413us (3567us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.330215:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1631 00000100:00100000:2.0:1788845047.332247:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1205 00000100:00100000:2.0:1788845047.332274:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641830400:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.332294:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.332313:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af173 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5c2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.332369:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a16a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660414208:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.332448:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.335388:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a16a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660414208:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.335429:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af173 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c2 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.335455:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af173 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c2 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.335481:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845047.335610:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641830400:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3340us (9066us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.339141:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1205 00000100:00100000:1.0:1788845047.341584:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.341719:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641830528 00000100:00100000:3.0:1788845047.341729:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641830528 00000100:00100000:3.0:1788845047.341787:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1632 00000100:00100000:3.0:1788845047.341796:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641830528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.341819:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.341840:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5c9 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.341873:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.341899:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.341938:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.341957:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 12 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.341971:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660414336:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845047.342049:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.344930:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660414336:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845047.344966:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.344986:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.344990:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.344994:0:11036:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.345005:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845047.345054:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641830528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3259us (3478us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.345062:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1632 00000100:00100000:2.0:1788845047.349016:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.349264:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641830656 00000100:00100000:3.0:1788845047.349276:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641830656 00010000:00010000:3.0:1788845047.349317:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 13 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.349369:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 559 00000100:00100000:3.0:1788845047.349379:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a16a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641830656:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845047.349394:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845047.349419:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 13 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.349438:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 13 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.349459:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af0d9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa544 expref: 13 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.349485:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.349492:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.349503:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.349512:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.349550:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.349568:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660414464:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845047.349651:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.352262:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.353207:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660414464:192.168.201.29@tcp:105: 00010000:00010000:2.0:1788845047.353233:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845047.353307:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a16a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641830656:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3928us (4300us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845047.353318:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 559 00000100:00100000:0.0:1788845047.354232:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641830784 00000100:00100000:0.0:1788845047.354242:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641830784 00000100:00100000:0.0:1788845047.354303:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1206 00000100:00100000:0.0:1788845047.354312:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641830784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.354330:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.354348:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af181 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5d0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.354404:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660414592:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.354472:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.356656:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660414592:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.356693:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af181 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5d0 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.356716:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af181 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5d0 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.356729:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.356779:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641830784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2468us (4526us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.356787:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1206 00000100:00100000:3.0:1788845047.360371:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.360647:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641830912 00000100:00100000:1.0:1788845047.360656:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641830912 00002000:00100000:1.0:1788845047.360705:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a10a80 x1875739641830912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 504/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.360717:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a10a80 with x1875739641830912 00000100:00100000:1.0:1788845047.360794:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1050 00000100:00100000:1.0:1788845047.360804:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641830912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.375715:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641830912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 14898us (15339us total) trans 0 rc 282624/282624 00000100:00100000:1.0:1788845047.375731:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1050 00002000:00100000:1.0:1788845047.375745:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a10a80 x1875739641830912/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 504/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 282624/282624 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.375763:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a10a80 with x1875739641830912 00000100:00100000:2.0:1788845047.379938:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.380032:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641831040 00000100:00100000:0.0:1788845047.380044:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641831040 00000100:00100000:0.0:1788845047.380112:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1207 00000100:00100000:0.0:1788845047.380279:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641831040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.380299:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.380317:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af188 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5d7 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.380381:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660414720:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.380484:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.382986:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660414720:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.383028:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af188 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5d7 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.383061:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af188 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5d7 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.383075:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.383143:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641831040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2869us (3215us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.383151:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1207 00010000:00010000:1.0:1788845047.383921:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845047.387214:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.387408:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641831168 00000100:00100000:2.0:1788845047.387418:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641831168 00000100:00100000:2.0:1788845047.387471:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1208 00000100:00100000:2.0:1788845047.387480:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641831168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.387498:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.387515:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af18f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5de expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.387571:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aeaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660414848:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.387791:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845047.392780:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aeaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660414848:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845047.392822:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af18f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5de expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.392848:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af18f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5de expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.392863:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865c00, rc 0) 00000100:00100000:1.0:1788845047.393130:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641831168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5649us (5925us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845047.393140:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1208 00000100:00100000:2.0:1788845047.400994:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.401256:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641831296 00000100:00100000:3.0:1788845047.401266:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641831296 00002000:00100000:3.0:1788845047.401319:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae9880 x1875739641831296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 504/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.401331:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae9880 with x1875739641831296 00000100:00100000:3.0:1788845047.401371:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1051 00000100:00100000:3.0:1788845047.401381:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641831296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.403988:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641831296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 2597us (2995us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.404004:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1051 00002000:00100000:3.0:1788845047.404016:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae9880 x1875739641831296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 504/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.404026:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae9880 with x1875739641831296 00000100:00100000:2.0:1788845047.407686:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.407810:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641831424 00000100:00100000:0.0:1788845047.407819:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641831424 00000100:00100000:0.0:1788845047.407933:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1209 00000100:00100000:0.0:1788845047.407943:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641831424:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.407961:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.407979:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af196 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5e5 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.408032:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660414976:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.408100:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.410158:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660414976:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.410285:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af196 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5e5 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.410313:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af196 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5e5 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.410332:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.410468:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641831424:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2524us (2788us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.410477:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1209 00000100:00100000:1.0:1788845047.414363:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.414647:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641831552 00000100:00100000:0.0:1788845047.414657:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641831552 00000100:00100000:0.0:1788845047.414769:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1210 00000100:00100000:0.0:1788845047.414778:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641831552:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.414797:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.414814:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af19d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5ec expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.414866:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660415104:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.415129:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.419572:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660415104:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.419624:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af19d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5ec expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.419650:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af19d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5ec expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.419665:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845047.419771:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641831552:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4993us (5417us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.419780:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1210 00000100:00100000:2.0:1788845047.426388:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.426529:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641831680 00000100:00100000:3.0:1788845047.426541:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641831680 00002000:00100000:3.0:1788845047.426590:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a039cdb050 x1875739641831680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.426610:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a039cdb050 with x1875739641831680 00000100:00100000:3.0:1788845047.426698:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1052 00000100:00100000:3.0:1788845047.426710:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a039cdb050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641831680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.427019:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a039cdb050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641831680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 309us (640us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.427027:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1052 00002000:00100000:3.0:1788845047.427034:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a039cdb050 x1875739641831680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.427043:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a039cdb050 with x1875739641831680 00000100:00100000:1.0:1788845047.432562:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.432714:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641831808 00000100:00100000:2.0:1788845047.432726:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641831808 00000100:00100000:2.0:1788845047.432859:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1211 00000100:00100000:2.0:1788845047.432872:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641831808:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.432929:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.432950:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287af1a4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5f3 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.433020:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660415232:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.433111:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.434582:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660415232:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.434627:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287af1a4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5f3 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.434651:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287af1a4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5f3 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.434663:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864200, rc 0) 00000100:00100000:0.0:1788845047.434753:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641831808:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1883us (2198us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.434761:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1211 00000100:00100000:3.0:1788845047.436666:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.436773:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641831936 00000100:00100000:2.0:1788845047.436780:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641831936 00000100:00100000:2.0:1788845047.436824:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1212 00000100:00100000:2.0:1788845047.436831:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641831936:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.436847:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.436861:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287af1ab lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa5fa expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.436926:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660415360:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.437021:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.438674:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660415360:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.438705:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287af1ab lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5fa expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.438724:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028e13120/0xb2187301287af1ab lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5fa expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.438735:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865400, rc 0) 00000100:00100000:2.0:1788845047.438777:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641831936:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1945us (2115us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.438785:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1212 00000100:00100000:2.0:1788845047.440741:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.440860:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641832064 00000100:00100000:0.0:1788845047.440870:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641832064 00002000:00100000:0.0:1788845047.441000:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae8380 x1875739641832064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.441008:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae8380 with x1875739641832064 00000100:00100000:0.0:1788845047.441038:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1053 00000100:00100000:0.0:1788845047.441045:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641832064:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.441380:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641832064:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 333us (646us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845047.441388:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1053 00002000:00100000:0.0:1788845047.441394:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae8380 x1875739641832064/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.441406:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae8380 with x1875739641832064 00000100:00100000:3.0:1788845047.443342:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.443687:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641832192 00000100:00100000:0.0:1788845047.443697:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641832192 00000100:00100000:0.0:1788845047.443744:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1213 00000100:00100000:0.0:1788845047.443754:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a12a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641832192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.443772:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.443788:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1b2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa601 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.443831:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660415488:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.443915:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.445218:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660415488:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.445247:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1b2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa601 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.445268:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1b2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa601 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.445282:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845047.445329:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a12a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641832192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1575us (1990us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.445337:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1213 00000100:00100000:2.0:1788845047.447334:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.447431:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641832320 00000100:00100000:0.0:1788845047.447440:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641832320 00000100:00100000:0.0:1788845047.447535:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1214 00000100:00100000:0.0:1788845047.447546:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aeb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641832320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.447570:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.447590:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1b9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa608 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.447665:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660415616:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.447757:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.449436:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660415616:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.449464:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1b9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa608 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.449485:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1b9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa608 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.449500:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.449549:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aeb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641832320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2006us (2221us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.449555:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1214 00000100:00100000:1.0:1788845047.451139:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.451218:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641832448 00000100:00100000:3.0:1788845047.451226:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641832448 00002000:00100000:3.0:1788845047.451270:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f20000 x1875739641832448/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.451278:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20000 with x1875739641832448 00000100:00100000:3.0:1788845047.451302:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1054 00000100:00100000:3.0:1788845047.451308:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641832448:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.451559:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641832448:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 249us (426us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.451565:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1054 00002000:00100000:3.0:1788845047.451570:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a034f20000 x1875739641832448/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.451575:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a034f20000 with x1875739641832448 00000100:00100000:2.0:1788845047.453192:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.453233:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641832576 00000100:00100000:0.0:1788845047.453240:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641832576 00000100:00100000:0.0:1788845047.453401:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1215 00000100:00100000:0.0:1788845047.453411:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641832576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.453427:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.453444:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1c0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa60f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.453494:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660415744:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.453564:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.454813:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660415744:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.454838:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1c0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa60f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.454854:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1c0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa60f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.454864:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845047.454944:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641832576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1535us (1758us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.454952:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1215 00000100:00100000:1.0:1788845047.456805:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.456867:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641832704 00000100:00100000:2.0:1788845047.456875:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641832704 00000100:00100000:2.0:1788845047.456978:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1216 00000100:00100000:2.0:1788845047.456988:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641832704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.457006:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.457023:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1c7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa616 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.457069:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aea300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660415872:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.457149:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.458448:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aea300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660415872:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.458480:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1c7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa616 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.458501:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1c7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa616 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.458515:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:2.0:1788845047.458558:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641832704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1571us (1758us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.458566:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1216 00000100:00100000:1.0:1788845047.460052:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.460278:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641832832 00000100:00100000:3.0:1788845047.460287:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641832832 00002000:00100000:3.0:1788845047.460329:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482d80 x1875739641832832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.460337:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482d80 with x1875739641832832 00000100:00100000:3.0:1788845047.460393:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1055 00000100:00100000:3.0:1788845047.460403:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641832832:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.460657:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641832832:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 256us (611us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.460663:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1055 00002000:00100000:3.0:1788845047.460668:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482d80 x1875739641832832/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.460673:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482d80 with x1875739641832832 00000100:00100000:1.0:1788845047.462495:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.462535:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641832960 00000100:00100000:2.0:1788845047.462544:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641832960 00000100:00100000:2.0:1788845047.462605:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1217 00000100:00100000:2.0:1788845047.462614:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641832960:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.462629:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.462643:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287af1ce lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa61d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.462691:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aea300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660416000:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.462754:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.464082:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aea300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660416000:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.464127:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287af1ce lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa61d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.464145:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287af1ce lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa61d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.464155:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845047.464195:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641832960:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1582us (1705us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.464204:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1217 00000100:00100000:1.0:1788845047.465989:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.466048:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641833088 00000100:00100000:0.0:1788845047.466056:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641833088 00000100:00100000:0.0:1788845047.466114:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1218 00000100:00100000:0.0:1788845047.466121:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641833088:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.466138:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.466152:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1d5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa624 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.466195:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660416128:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.466270:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.467561:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660416128:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.467587:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1d5 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa624 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.467618:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af1d5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa624 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.467628:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845047.467669:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641833088:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1547us (1686us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.467676:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1218 00000100:00100000:1.0:1788845047.469032:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.469120:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641833216 00000100:00100000:3.0:1788845047.469130:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641833216 00002000:00100000:3.0:1788845047.469177:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482a00 x1875739641833216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.469189:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482a00 with x1875739641833216 00000100:00100000:3.0:1788845047.469319:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1056 00000100:00100000:3.0:1788845047.469328:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641833216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.469574:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641833216:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 247us (546us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.469628:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1056 00002000:00100000:3.0:1788845047.469635:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482a00 x1875739641833216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.469643:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482a00 with x1875739641833216 00000100:00100000:1.0:1788845047.471109:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.471147:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641833344 00000100:00100000:0.0:1788845047.471155:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641833344 00000100:00100000:0.0:1788845047.471211:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1219 00000100:00100000:0.0:1788845047.471220:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641833344:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.471236:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.471252:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1dc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa62b expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.471448:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660416256:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.471521:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.472671:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660416256:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.472702:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1dc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa62b expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.472722:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1dc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa62b expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.472732:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.472773:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641833344:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1554us (1679us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.472780:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1219 00000100:00100000:1.0:1788845047.476285:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.476659:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641833472 00000100:00100000:2.0:1788845047.476669:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641833472 00000100:00100000:2.0:1788845047.476758:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1220 00000100:00100000:2.0:1788845047.476769:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641833472:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.476787:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.476806:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287af1e3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa632 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.476861:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aea300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660416384:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.476963:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.479651:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aea300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660416384:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.479691:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287af1e3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa632 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.479718:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001c8d6abb/0xb2187301287af1e3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa632 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.479731:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845047.479793:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641833472:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3025us (3518us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.479801:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1220 00000100:00100000:1.0:1788845047.483844:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.484000:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641833600 00000100:00100000:0.0:1788845047.484010:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641833600 00002000:00100000:0.0:1788845047.484056:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482680 x1875739641833600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.484068:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482680 with x1875739641833600 00000100:00100000:0.0:1788845047.484109:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1057 00000100:00100000:0.0:1788845047.484128:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641833600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.484386:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010482680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641833600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 259us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845047.484394:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1057 00002000:00100000:0.0:1788845047.484401:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a010482680 x1875739641833600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.484409:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a010482680 with x1875739641833600 00000100:00100000:1.0:1788845047.488029:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.488110:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641833728 00000100:00100000:2.0:1788845047.488129:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641833728 00000100:00100000:2.0:1788845047.488190:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1221 00000100:00100000:2.0:1788845047.488201:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641833728:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.488223:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.488240:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1ea lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa639 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.488307:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660416512:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.488402:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.490667:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660416512:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.490703:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1ea lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa639 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.490727:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1ea lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa639 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.490740:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:2.0:1788845047.490781:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641833728:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2581us (2761us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.490789:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1221 00000100:00100000:1.0:1788845047.493744:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.493822:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641833856 00000100:00100000:0.0:1788845047.493831:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641833856 00000100:00100000:0.0:1788845047.493880:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1222 00000100:00100000:0.0:1788845047.493911:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641833856:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.493931:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.493946:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1f1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa640 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.493998:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660416640:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.494068:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.496141:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660416640:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.496180:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1f1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa640 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.496204:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af1f1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa640 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.496217:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.496590:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641833856:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2678us (2852us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.496607:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1222 00000100:00100000:1.0:1788845047.500226:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.500390:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641833984 00000100:00100000:0.0:1788845047.500401:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641833984 00002000:00100000:0.0:1788845047.500447:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f776680 x1875739641833984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.500459:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f776680 with x1875739641833984 00000100:00100000:0.0:1788845047.500502:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1058 00000100:00100000:0.0:1788845047.500512:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641833984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.500804:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641833984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 293us (588us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845047.500812:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1058 00002000:00100000:0.0:1788845047.500819:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f776680 x1875739641833984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.500826:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f776680 with x1875739641833984 00000100:00100000:1.0:1788845047.503845:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.504084:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641834112 00000100:00100000:2.0:1788845047.504094:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641834112 00000100:00100000:2.0:1788845047.504190:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1223 00000100:00100000:2.0:1788845047.504201:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641834112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.504220:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.504238:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1f8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa647 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.504315:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660416768:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.504397:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.507202:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788845047.507225:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a11880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660416896:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845047.507293:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845047.507385:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660416896 00000100:00100000:3.0:1788845047.507393:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660416896 00000100:00100000:3.0:1788845047.507465:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 86 00000100:00100000:3.0:1788845047.507481:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660416896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788845047.507520:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788845047.507569:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aeb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660416896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 96us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.507577:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 86 00000100:00080000:2.0:1788845047.507648:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845047.507684:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a11880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660416896:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845047.513945:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660416768:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845047.513987:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1f8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa647 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.514011:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c037621/0xb2187301287af1f8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa647 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.514025:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:3.0:1788845047.514128:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641834112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9927us (10291us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.514137:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1223 00000100:00100000:1.0:1788845047.517531:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.517663:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641834240 00000100:00100000:2.0:1788845047.517674:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641834240 00000100:00100000:2.0:1788845047.517735:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1224 00000100:00100000:2.0:1788845047.517746:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641834240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.517765:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.517784:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287af1ff lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa64e expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.517843:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aeb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660417024:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.518015:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.522039:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aeb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660417024:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.522079:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287af1ff lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa64e expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.522118:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4acf5d4/0xb2187301287af1ff lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa64e expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.522133:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864400, rc 0) 00000100:00100000:2.0:1788845047.522230:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641834240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4485us (4708us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.522238:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1224 00000100:00100000:3.0:1788845047.526336:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.526475:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641834368 00000100:00100000:0.0:1788845047.526486:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641834368 00002000:00100000:0.0:1788845047.526538:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a13100 x1875739641834368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.526549:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a13100 with x1875739641834368 00000100:00100000:0.0:1788845047.526588:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1059 00000100:00100000:0.0:1788845047.526610:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a13100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641834368:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.526965:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a13100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641834368:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 357us (638us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845047.526974:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1059 00002000:00100000:0.0:1788845047.526981:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a13100 x1875739641834368/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.526988:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a13100 with x1875739641834368 00000100:00100000:1.0:1788845047.531548:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.531650:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641834496 00000100:00100000:0.0:1788845047.531661:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641834496 00000100:00100000:0.0:1788845047.531721:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1225 00000100:00100000:0.0:1788845047.531731:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641834496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.531750:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.531769:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af206 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa655 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.531834:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660417152:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.531970:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18240 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.534141:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660417152:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.534182:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af206 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa655 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.534345:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af206 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa655 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.534355:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.534453:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641834496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2723us (2914us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.534462:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1225 00000100:00100000:3.0:1788845047.537094:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.537218:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641834624 00000100:00100000:1.0:1788845047.537228:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641834624 00002000:00100000:1.0:1788845047.537280:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a11f80 x1875739641834624/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.537292:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a11f80 with x1875739641834624 00000100:00100000:1.0:1788845047.537330:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1060 00000100:00100000:1.0:1788845047.537339:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a11f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641834624:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845047.540448:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 281699 -> 581632 00000004:00010000:2.0:1788845047.540458:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00000004:00010000:2.0:1788845047.540461:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845047 -> 1788845043 00000004:00010000:2.0:1788845047.540463:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00000004:00010000:2.0:1788845047.540465:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 198 -> 198 00000100:00100000:2.0:1788845047.540552:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a11f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641834624:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3209us (3462us total) trans 4294967873 rc 0/0 00000100:00100000:2.0:1788845047.540563:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1060 00002000:00100000:2.0:1788845047.540576:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a11f80 x1875739641834624/t4294967873(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/448 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.540586:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a11f80 with x1875739641834624 00000100:00100000:1.0:1788845047.544566:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.544676:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641834752 00000100:00100000:2.0:1788845047.544685:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641834752 00000100:00100000:2.0:1788845047.544747:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1633 00000100:00100000:2.0:1788845047.544758:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f776d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641834752:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:0.0:1788845047.568635:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967873 is committed 00000001:00080000:0.0:1788845047.568655:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967872 is committed 00000100:00100000:3.0:1788845047.568836:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f776d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641834752:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 24075us (24275us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.568849:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1633 00000100:00100000:2.0:1788845047.571991:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.572101:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641834880 00000100:00100000:0.0:1788845047.572174:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641834880 00000100:00100000:0.0:1788845047.572274:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1226 00000100:00100000:0.0:1788845047.572284:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aeb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641834880:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.572302:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.572320:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af20d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa65c expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.572383:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660417280:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.572620:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.574809:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660417280:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.574847:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af20d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa65c expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.574873:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ec9513d4/0xb2187301287af20d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa65c expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.574920:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845047.575018:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aeb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641834880:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2734us (3037us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.575027:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1226 00000100:00100000:2.0:1788845047.577140:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.577324:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641835136 00000100:00100000:3.0:1788845047.577334:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641835136 00000100:00100000:3.0:1788845047.577418:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 128 00000100:00100000:3.0:1788845047.577430:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641835136:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845047.577500:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641835136:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 72us (368us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.577507:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 128 00000100:00100000:1.0:1788845047.578011:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.579521:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641835008 00000100:00100000:2.0:1788845047.579532:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641835008 00000100:00100000:2.0:1788845047.579602:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 287 00000100:00100000:2.0:1788845047.579613:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641835008:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845047.579689:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641835008:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 78us (1679us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845047.579697:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 287 00000100:00100000:2.0:1788845047.581488:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.581564:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641835264 00000100:00100000:0.0:1788845047.581573:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641835264 00000100:00100000:0.0:1788845047.581635:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1227 00000100:00100000:0.0:1788845047.581645:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641835264:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.581663:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.581681:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af214 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa663 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.581735:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660417408:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.581802:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.584441:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660417408:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.584490:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af214 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa663 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.584515:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af214 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa663 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.584529:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845047.584582:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641835264:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2938us (3112us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.584590:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1227 00000100:00100000:1.0:1788845047.587269:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.587410:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641835520 00000100:00100000:2.0:1788845047.587421:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641835520 00000100:00100000:2.0:1788845047.587487:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1634 00000100:00100000:2.0:1788845047.587498:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641835520:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:2.0:1788845047.587554:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845047.587581:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a000030a00/0xb2187301287af165 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5b4 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.587611:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.587616:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845047.587860:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.587874:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.587906:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.587945:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.587963:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 14 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.587976:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660417536:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845047.588058:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.590771:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.590923:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660417536:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845047.590951:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.591050:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641835648 00000100:00100000:3.0:1788845047.591057:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641835648 00010000:00010000:3.0:1788845047.591089:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 15 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.591215:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 560 00000100:00100000:3.0:1788845047.591227:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641835648:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845047.591242:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845047.591249:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 15 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.591280:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 15 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.591303:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af11f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa575 expref: 15 pid: 5839 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.591316:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.591321:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.591330:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.591342:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.591369:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845047.591425:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641835648:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 199us (663us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.591432:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 560 00010000:00010000:1.0:1788845047.591494:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.591502:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845047.591629:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967874 00000004:00010000:1.0:1788845047.591679:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 581632 -> 581632 00000004:00010000:1.0:1788845047.591683:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00000004:00010000:1.0:1788845047.591686:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:1.0:1788845047.591688:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00000004:00010000:1.0:1788845047.591690:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 198 -> 326 00000004:00010000:1.0:1788845047.591698:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000029a0fc49 reply state 00000000cad85980 transno 4294967875 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.591706:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845047.591769:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641835520:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4273us (4526us total) trans 4294967875 rc 0/0 00000100:00100000:1.0:1788845047.591776:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1634 00000020:00080000:1.0:1788845047.591780:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788845047.596807:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.597452:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641835776 00000100:00100000:2.0:1788845047.597463:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641835776 00000100:00100000:2.0:1788845047.597603:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1635 00000100:00100000:2.0:1788845047.597614:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641835776:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.597637:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.597658:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa66a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.597687:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.597698:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.597727:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.597744:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 13 pid: 11036 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.597756:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aeb480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660417664:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845047.597833:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788845047.597968:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.597977:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.597984:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.597990:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845047.597995:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.598003:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.598017:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845047.598019:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845047.598022:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.598032:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865400/0xb2187301287af21b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.602490:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.602634:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641835904 00000100:00100000:0.0:1788845047.602643:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641835904 00010000:00010000:0.0:1788845047.602682:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 14 pid: 11036 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.602724:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 561 00000100:00100000:0.0:1788845047.602734:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641835904:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845047.602748:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845047.602774:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 14 pid: 11036 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.602792:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 14 pid: 11036 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.602818:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845047.602825:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.602835:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.602843:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.602918:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.602934:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660417792:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845047.603012:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18b40 going to sleep for 11 seconds 00010000:00010000:3.0:1788845047.605421:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af17a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa5c9 expref: 14 pid: 11036 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.605448:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aeb480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660417664:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845047.605555:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 12 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.605573:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.605577:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.605581:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:1.0:1788845047.605995:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660417792:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845047.606008:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845047.606061:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:2370:x1875739641835904:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 3328us (3579us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845047.606072:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 561 00000100:00100000:2.0:1788845047.607544:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641835776:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9923us (10740us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845047.607556:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1635 00000100:00100000:0.0:1788845047.610703:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.610796:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.610863:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641836032 00000100:00100000:3.0:1788845047.610874:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641836032 00002000:00010000:3.0:1788845047.610988:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845047.610994:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aa9c00 with x1875739641836032 00000100:00100000:3.0:1788845047.611073:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1061 00000100:00100000:3.0:1788845047.611085:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641836032:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:3.0:1788845047.611400:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 581632 -> 244318 00000004:00010000:3.0:1788845047.611404:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00000004:00010000:3.0:1788845047.611407:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:3.0:1788845047.611410:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00000004:00010000:3.0:1788845047.611412:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845047.611498:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641836032:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 414us (700us total) trans 4294967876 rc 0/0 00000100:00100000:3.0:1788845047.611507:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1061 00002000:00010000:3.0:1788845047.611510:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788845047.611513:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aa9c00 with x1875739641836032 00000100:00100000:2.0:1788845047.612447:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641836160 00000100:00100000:2.0:1788845047.612457:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641836160 00002000:00010000:2.0:1788845047.612502:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845047.612508:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aab800 with x1875739641836160 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845047.612518:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aab800. 00000100:00100000:2.0:1788845047.612554:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 636 00000100:00100000:2.0:1788845047.612562:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+6:2368:x1875739641836160:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845047.612943:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1291195 -> 0 00002000:00010000:2.0:1788845047.612947:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00002000:00010000:2.0:1788845047.612950:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845047 -> 1788845046 00002000:00010000:2.0:1788845047.612952:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00002000:00010000:2.0:1788845047.612955:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 383 -> 137 00000100:00100000:2.0:1788845047.613005:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2368:x1875739641836160:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 443us (2316us total) trans 4294967516 rc 0/0 00000100:00100000:2.0:1788845047.613013:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 636 00002000:00010000:2.0:1788845047.613017:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1788845047.613020:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03aab800 with x1875739641836160 ext(0->18446744073709551615) 00002000:00010000:2.0:1788845047.613025:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03aab800. 00000100:00100000:0.0:1788845047.617240:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.617391:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641836288 00000100:00100000:1.0:1788845047.617400:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641836288 00000100:00100000:1.0:1788845047.617560:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1636 00000100:00100000:1.0:1788845047.617572:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:14891:x1875739641836288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.617606:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.617628:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287af229 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa671 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.617718:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.617732:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.617744:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845047.617770:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 244318 00000004:00200000:1.0:1788845047.617775:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845047.617793:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.617819:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1800/0xb2187301287af229 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.617835:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.617844:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845047.617847:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845047.617850:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0000, rc 0) 00000100:00100000:1.0:1788845047.619232:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:14891:x1875739641836288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1653us (1992us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845047.619244:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1636 00000100:00100000:2.0:1788845047.622699:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.622769:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641836416 00000100:00100000:0.0:1788845047.622778:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641836416 00000100:00100000:0.0:1788845047.622849:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1062 00000100:00100000:0.0:1788845047.622860:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+14:2369:x1875739641836416:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845047.623136:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 244318 -> 244318 00000004:00010000:0.0:1788845047.623140:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00000004:00010000:0.0:1788845047.623143:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:0.0:1788845047.623146:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00000004:00010000:0.0:1788845047.623148:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845047.623403:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641836416:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 544us (715us total) trans 4294967877 rc 0/0 00000100:00100000:0.0:1788845047.623412:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1062 00000100:00100000:2.0:1788845047.627089:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.627240:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641836544 00000100:00100000:0.0:1788845047.627250:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641836544 00000100:00100000:0.0:1788845047.627357:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1063 00000100:00100000:0.0:1788845047.627368:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641836544:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845047.627637:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 244318 -> 244318 00000004:00010000:0.0:1788845047.627642:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00000004:00010000:0.0:1788845047.627645:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845043 -> 1788845043 00000004:00010000:0.0:1788845047.627647:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00000004:00010000:0.0:1788845047.627651:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845047.627726:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641836544:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 359us (644us total) trans 4294967878 rc 0/0 00000100:00100000:0.0:1788845047.627733:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1063 00000100:00100000:1.0:1788845047.631049:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.631096:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.631272:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641836672 00000100:00100000:3.0:1788845047.631283:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641836672 00000100:00100000:3.0:1788845047.631346:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1637 00000100:00100000:3.0:1788845047.631357:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2368:x1875739641836672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.631380:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.631401:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af237 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa678 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.631438:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.631491:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a11880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660417920:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.631567:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22cc0 going to sleep for 11 seconds 00000004:00010000:3.0:1788845047.633532:0:5839:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845043 -> 1788845047 00000100:00100000:3.0:1788845047.633564:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a11880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660417920:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845047.633585:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:3.0:1788845047.633614:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af237 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa678 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.633636:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.633639:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.633645:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af237 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa678 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.633660:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845047.633720:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2368:x1875739641836672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2364us (2682us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.633729:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1637 00000100:00100000:2.0:1788845047.634616:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641836800 00000100:00100000:2.0:1788845047.634625:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641836800 00000100:00100000:2.0:1788845047.634681:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1228 00000100:00100000:2.0:1788845047.634691:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641836800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.634709:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.634725:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af23e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa67f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.634779:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aead80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660418048:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.634854:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845047.637508:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845046 -> 1788845047 00000100:00100000:2.0:1788845047.637541:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aead80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660418048:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.637576:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af23e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa67f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.637611:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af23e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa67f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.637624:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845047.637691:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641836800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3000us (6593us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.637700:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1228 00000100:00100000:0.0:1788845047.641296:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.641368:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641836928 00000100:00100000:2.0:1788845047.641377:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641836928 00000100:00100000:2.0:1788845047.641438:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1638 00000100:00100000:2.0:1788845047.641450:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:14891:x1875739641836928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.641482:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.641504:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af245 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa686 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.641536:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.641547:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af245 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa686 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.641580:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.641608:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 15 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.641622:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aead80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660418176:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845047.641711:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845047.644609:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.644745:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641837056 00000100:00100000:0.0:1788845047.644754:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641837056 00010000:00010000:0.0:1788845047.644792:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 16 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.644838:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 562 00000100:00100000:0.0:1788845047.644848:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641837056:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845047.644864:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845047.644932:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 16 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.644952:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 16 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.644978:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845047.644984:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af245 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa686 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.644996:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af245 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa686 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.645005:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af245 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa686 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.645044:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af245 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa686 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.645060:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660418304:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845047.645142:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18900 going to sleep for 11 seconds 00010000:00010000:2.0:1788845047.645328:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af222 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa66a expref: 16 pid: 5841 timeout: 536 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.645345:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aead80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660418176:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845047.645370:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af245 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa686 expref: 14 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.645381:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.645384:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845047.645387:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864000, rc 0) 00000100:00100000:2.0:1788845047.645417:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641836928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3970us (4131us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845047.645425:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1638 00000100:00100000:3.0:1788845047.650423:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.650508:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641837184 00000100:00100000:2.0:1788845047.650518:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641837184 00002000:00100000:2.0:1788845047.650565:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a11880 x1875739641837184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.650583:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a11880 with x1875739641837184 00000100:00100000:2.0:1788845047.650648:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1064 00000100:00100000:2.0:1788845047.650658:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a11880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641837184:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.650921:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a11880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641837184:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 263us (506us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.650929:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1064 00002000:00100000:2.0:1788845047.650936:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a11880 x1875739641837184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.650944:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a11880 with x1875739641837184 00000100:00100000:0.0:1788845047.651795:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13304:1875739660418304:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845047.651819:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845047.651865:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641837056:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7018us (7277us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845047.651875:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 562 00000100:00100000:3.0:1788845047.654651:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.654759:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641837312 00000100:00100000:1.0:1788845047.654770:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641837312 00002000:00100000:1.0:1788845047.654827:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a13480 x1875739641837312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.654837:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a13480 with x1875739641837312 00000100:00100000:1.0:1788845047.654876:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1065 00000100:00100000:1.0:1788845047.654918:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a13480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641837312:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.655217:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a13480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641837312:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 331us (577us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.655225:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1065 00002000:00100000:1.0:1788845047.655234:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a13480 x1875739641837312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.655242:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a13480 with x1875739641837312 00000100:00100000:1.0:1788845047.659248:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.659434:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641837440 00000100:00100000:2.0:1788845047.659445:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641837440 00002000:00100000:2.0:1788845047.659505:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dc700 x1875739641837440/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.659516:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dc700 with x1875739641837440 00000100:00100000:2.0:1788845047.659549:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1066 00000100:00100000:2.0:1788845047.659559:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641837440:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.659840:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641837440:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (600us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.659848:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1066 00002000:00100000:2.0:1788845047.659855:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dc700 x1875739641837440/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.659863:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dc700 with x1875739641837440 00000100:00100000:2.0:1788845047.664665:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.664780:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641837568 00000100:00100000:1.0:1788845047.664791:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641837568 00002000:00100000:1.0:1788845047.664839:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03aead80 x1875739641837568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.664849:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aead80 with x1875739641837568 00000100:00100000:1.0:1788845047.665013:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1067 00000100:00100000:1.0:1788845047.665024:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641837568:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.665416:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641837568:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 392us (759us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845047.665424:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1067 00002000:00100000:1.0:1788845047.665433:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03aead80 x1875739641837568/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.665442:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aead80 with x1875739641837568 00000100:00100000:1.0:1788845047.668691:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.668813:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641837696 00000100:00100000:2.0:1788845047.668822:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641837696 00002000:00100000:2.0:1788845047.668877:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5ddf80 x1875739641837696/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.668912:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5ddf80 with x1875739641837696 00000100:00100000:2.0:1788845047.668944:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1068 00000100:00100000:2.0:1788845047.668951:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641837696:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.669195:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641837696:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 241us (509us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.669203:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1068 00002000:00100000:2.0:1788845047.669210:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5ddf80 x1875739641837696/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.669217:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5ddf80 with x1875739641837696 00000100:00100000:2.0:1788845047.672311:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.672451:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641837824 00000100:00100000:0.0:1788845047.672461:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641837824 00002000:00100000:0.0:1788845047.672518:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae8e00 x1875739641837824/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.672529:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae8e00 with x1875739641837824 00000100:00100000:0.0:1788845047.672566:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1069 00000100:00100000:0.0:1788845047.672575:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641837824:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.672843:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641837824:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (540us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.672851:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1069 00002000:00100000:0.0:1788845047.672858:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae8e00 x1875739641837824/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.672867:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae8e00 with x1875739641837824 00000100:00100000:1.0:1788845047.676475:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.676616:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641837952 00000100:00100000:2.0:1788845047.676624:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641837952 00002000:00100000:2.0:1788845047.676663:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dce00 x1875739641837952/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.676671:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dce00 with x1875739641837952 00000100:00100000:2.0:1788845047.676700:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1070 00000100:00100000:2.0:1788845047.676706:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641837952:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.677041:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641837952:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 333us (580us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.677049:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1070 00002000:00100000:2.0:1788845047.677055:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dce00 x1875739641837952/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.677063:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dce00 with x1875739641837952 00000100:00100000:3.0:1788845047.680493:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.680609:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641838080 00000100:00100000:0.0:1788845047.680620:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641838080 00002000:00100000:0.0:1788845047.680671:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a10700 x1875739641838080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.680683:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a10700 with x1875739641838080 00000100:00100000:0.0:1788845047.680725:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1071 00000100:00100000:0.0:1788845047.680735:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641838080:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.681028:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641838080:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 294us (543us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.681036:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1071 00002000:00100000:0.0:1788845047.681042:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a10700 x1875739641838080/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.681056:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a10700 with x1875739641838080 00000100:00100000:1.0:1788845047.684038:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.684130:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641838208 00000100:00100000:3.0:1788845047.684139:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641838208 00002000:00100000:3.0:1788845047.684190:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5df100 x1875739641838208/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.684201:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5df100 with x1875739641838208 00000100:00100000:3.0:1788845047.684242:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1072 00000100:00100000:3.0:1788845047.684253:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641838208:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.684504:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641838208:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 251us (471us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.684510:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1072 00002000:00100000:3.0:1788845047.684516:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5df100 x1875739641838208/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.684521:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5df100 with x1875739641838208 00000100:00100000:2.0:1788845047.686663:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.686723:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641838336 00000100:00100000:1.0:1788845047.686732:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641838336 00000100:00100000:1.0:1788845047.686826:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1639 00000100:00100000:1.0:1788845047.686834:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641838336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.686851:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.686868:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af24c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa68d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.686972:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af253 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.686984:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af253 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.686994:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af253 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845047.687015:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 244318 00000004:00200000:1.0:1788845047.687018:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845047.687032:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af253 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.687050:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af24c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa68d expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.687063:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af253 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa68d expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.687069:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845047.687072:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845047.687075:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0a00, rc 0) 00000100:00100000:1.0:1788845047.687146:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641838336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 311us (492us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845047.687153:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1639 00000100:00100000:0.0:1788845047.689043:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.689159:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641838464 00000100:00100000:2.0:1788845047.689169:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641838464 00000100:00100000:2.0:1788845047.689222:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1229 00000100:00100000:2.0:1788845047.689232:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641838464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.689251:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.689266:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af25a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa694 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.689316:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aeb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660418432:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.689378:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.690588:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aeb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660418432:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.690635:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af25a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa694 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.690653:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af25a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa694 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.690664:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865c00, rc 0) 00000100:00100000:0.0:1788845047.690702:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641838464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1473us (1665us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.690707:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1229 00000100:00100000:1.0:1788845047.692788:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.692860:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641838592 00000100:00100000:3.0:1788845047.692869:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641838592 00000100:00100000:2.0:1788845047.693074:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1230 00000100:00100000:2.0:1788845047.693085:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641838592:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.693204:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.693220:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af261 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa69b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.693313:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660418560:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.693378:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.695243:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660418560:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.695347:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af261 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa69b expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.695373:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af261 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa69b expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.695387:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00000100:00100000:2.0:1788845047.695448:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641838592:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2364us (2665us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.695457:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1230 00000100:00100000:1.0:1788845047.697700:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.697925:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641838720 00000100:00100000:3.0:1788845047.697935:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641838720 00000100:00100000:3.0:1788845047.697986:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1640 00000100:00100000:3.0:1788845047.697997:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641838720:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.698020:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.698037:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6a2 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.698064:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845047.698073:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:3.0:1788845047.698078:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.698088:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.698101:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.698105:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845047.698159:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641838720:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 164us (467us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.698166:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1640 00000100:00100000:1.0:1788845047.700197:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.700343:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641838848 00000100:00100000:0.0:1788845047.700350:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641838848 00002000:00100000:0.0:1788845047.700397:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5df480 x1875739641838848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.700405:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5df480 with x1875739641838848 00000100:00100000:0.0:1788845047.700509:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1073 00000100:00100000:0.0:1788845047.700516:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641838848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.701024:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641838848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 507us (834us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845047.701030:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1073 00002000:00100000:0.0:1788845047.701037:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5df480 x1875739641838848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.701042:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5df480 with x1875739641838848 00000100:00100000:0.0:1788845047.706763:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.706927:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641838976 00000100:00100000:3.0:1788845047.706937:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641838976 00002000:00100000:3.0:1788845047.706991:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03aa8380 x1875739641838976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.707002:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aa8380 with x1875739641838976 00000100:00100000:3.0:1788845047.707039:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1074 00000100:00100000:3.0:1788845047.707048:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641838976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.707257:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641838976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 209us (503us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.707265:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1074 00002000:00100000:3.0:1788845047.707271:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03aa8380 x1875739641838976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.707280:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aa8380 with x1875739641838976 00000100:00100000:1.0:1788845047.713604:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.714061:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641839104 00000100:00100000:3.0:1788845047.714073:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641839104 00002000:00100000:3.0:1788845047.714147:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dd880 x1875739641839104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.714159:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dd880 with x1875739641839104 00000100:00100000:3.0:1788845047.714189:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1075 00000100:00100000:3.0:1788845047.714199:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.714656:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 456us (1071us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.714664:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1075 00002000:00100000:3.0:1788845047.714672:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03d5dd880 x1875739641839104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.714680:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03d5dd880 with x1875739641839104 00000100:00100000:1.0:1788845047.720928:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.721068:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641839232 00000100:00100000:3.0:1788845047.721079:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641839232 00002000:00100000:3.0:1788845047.721146:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f0000 x1875739641839232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.721159:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f0000 with x1875739641839232 00000100:00100000:3.0:1788845047.721199:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1076 00000100:00100000:3.0:1788845047.721210:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839232:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.721454:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 245us (535us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.721462:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1076 00002000:00100000:3.0:1788845047.721479:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f0000 x1875739641839232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.721488:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f0000 with x1875739641839232 00000100:00100000:1.0:1788845047.729376:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.729581:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641839360 00000100:00100000:3.0:1788845047.729604:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641839360 00002000:00100000:3.0:1788845047.729659:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f1c00 x1875739641839360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.729671:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1c00 with x1875739641839360 00000100:00100000:3.0:1788845047.729714:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1077 00000100:00100000:3.0:1788845047.729725:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.730012:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839360:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 288us (656us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.730020:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1077 00002000:00100000:3.0:1788845047.730028:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f1c00 x1875739641839360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.730036:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1c00 with x1875739641839360 00000100:00100000:1.0:1788845047.737450:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.737671:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641839488 00000100:00100000:3.0:1788845047.737682:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641839488 00002000:00100000:3.0:1788845047.737734:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f1f80 x1875739641839488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.737750:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1f80 with x1875739641839488 00000100:00100000:3.0:1788845047.737846:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1078 00000100:00100000:3.0:1788845047.737858:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.738183:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 326us (740us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.738192:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1078 00002000:00100000:3.0:1788845047.738199:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f1f80 x1875739641839488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.738207:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f1f80 with x1875739641839488 00000100:00100000:1.0:1788845047.743026:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.743127:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641839616 00000100:00100000:3.0:1788845047.743138:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641839616 00002000:00100000:3.0:1788845047.743190:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3800 x1875739641839616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.743201:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3800 with x1875739641839616 00000100:00100000:3.0:1788845047.743229:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1079 00000100:00100000:3.0:1788845047.743240:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641839616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.743510:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641839616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 270us (492us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.743517:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1079 00002000:00100000:3.0:1788845047.743524:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3800 x1875739641839616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.743531:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3800 with x1875739641839616 00000100:00100000:1.0:1788845047.756660:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.756805:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641839744 00000100:00100000:2.0:1788845047.756815:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641839744 00002000:00100000:2.0:1788845047.756866:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f2d80 x1875739641839744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.756878:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f2d80 with x1875739641839744 00000100:00100000:2.0:1788845047.756953:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1080 00000100:00100000:2.0:1788845047.756963:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845047.757460:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641839744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 496us (808us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845047.757804:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1080 00002000:00100000:2.0:1788845047.757812:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f2d80 x1875739641839744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845047.757821:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f2d80 with x1875739641839744 00000100:00100000:1.0:1788845047.761687:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.761818:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641839872 00000100:00100000:3.0:1788845047.761829:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641839872 00002000:00100000:3.0:1788845047.761879:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3b80 x1875739641839872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.761925:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3b80 with x1875739641839872 00000100:00100000:3.0:1788845047.761952:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1081 00000100:00100000:3.0:1788845047.761962:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641839872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.762409:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0351f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641839872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 447us (731us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845047.762418:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1081 00002000:00100000:3.0:1788845047.762426:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0351f3b80 x1875739641839872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:153/0 lens 488/4536 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.762434:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0351f3b80 with x1875739641839872 00000100:00100000:1.0:1788845047.769183:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.769285:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641840000 00000100:00100000:0.0:1788845047.769295:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641840000 00000100:00100000:0.0:1788845047.769361:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1231 00000100:00100000:0.0:1788845047.769381:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641840000:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.769400:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.769418:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af26f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6a9 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.769485:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aaa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660418688:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.769572:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c183c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.773416:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aaa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660418688:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.773456:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af26f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.773495:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af26f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a9 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.773509:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845047.773575:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641840000:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4195us (4402us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.773584:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1231 00000100:00100000:1.0:1788845047.784705:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.784810:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641840128 00000100:00100000:0.0:1788845047.784819:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641840128 00000100:00100000:0.0:1788845047.784879:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1232 00000100:00100000:0.0:1788845047.784919:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641840128:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.784939:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.784960:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af276 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6b0 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.785023:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aaa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660418816:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.785116:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c183c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.789148:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aaa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660418816:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.789188:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af276 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6b0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.789213:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af276 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6b0 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.789228:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845047.789291:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641840128:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4375us (4595us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.789301:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1232 00000100:00100000:1.0:1788845047.800010:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.800125:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641840256 00000100:00100000:2.0:1788845047.800136:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641840256 00000100:00100000:2.0:1788845047.800196:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1233 00000100:00100000:2.0:1788845047.800208:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641840256:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.800226:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.800244:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af27d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6b7 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.800314:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660418944:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.800406:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845047.802931:0:6684:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size: 0 -> 1233349 00000100:00100000:2.0:1788845047.802962:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660418944:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.802999:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af27d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6b7 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.803023:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af27d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6b7 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.803035:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00000100:00100000:2.0:1788845047.803141:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641840256:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2935us (3138us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.803150:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1233 00000100:00100000:1.0:1788845047.806996:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.807086:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641840384 00000100:00100000:2.0:1788845047.807103:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641840384 00000100:00100000:2.0:1788845047.807153:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1234 00000100:00100000:2.0:1788845047.807161:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641840384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.807179:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.807195:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af284 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6be expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.807251:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660419072:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.807325:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.810286:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660419072:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.810329:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af284 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6be expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.810355:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af284 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6be expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.810381:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845047.810445:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641840384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3282us (3456us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.810453:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1234 00000100:00100000:1.0:1788845047.815086:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.815236:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641840512 00000100:00100000:0.0:1788845047.815246:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641840512 00000100:00100000:0.0:1788845047.815565:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1235 00000100:00100000:0.0:1788845047.815577:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641840512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.815609:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.815629:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af28b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6c5 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.815690:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aaa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660419200:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.815784:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c183c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.819462:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aaa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660419200:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.819632:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af28b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6c5 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.819656:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f37ef69/0xb2187301287af28b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6c5 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.819669:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845047.819728:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641840512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4153us (4652us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.819736:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1235 00000100:00100000:1.0:1788845047.823692:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.823849:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641840640 00000100:00100000:0.0:1788845047.823859:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641840640 00000100:00100000:0.0:1788845047.823939:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1236 00000100:00100000:0.0:1788845047.823949:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641840640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.823968:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.823984:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287af292 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6cc expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.824047:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aaa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660419328:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.824134:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c183c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.827664:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aaa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660419328:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.827703:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287af292 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6cc expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.827727:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007753dc80/0xb2187301287af292 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6cc expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.827742:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845047.827793:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641840640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3846us (4110us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.827801:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1236 00000100:00100000:1.0:1788845047.831997:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.832150:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641840768 00000100:00100000:2.0:1788845047.832161:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641840768 00000100:00100000:2.0:1788845047.832221:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1237 00000100:00100000:2.0:1788845047.832233:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641840768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.832254:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.832273:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af299 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6d3 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.832337:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660419456:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.832436:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.835987:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660419456:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.836030:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af299 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6d3 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.836058:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af299 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6d3 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.836071:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845047.836197:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641840768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3964us (4209us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.836205:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1237 00000100:00100000:1.0:1788845047.840549:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.840647:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641840896 00000100:00100000:2.0:1788845047.840658:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641840896 00000100:00100000:2.0:1788845047.840762:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1238 00000100:00100000:2.0:1788845047.840772:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641840896:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.840793:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.840811:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af2a0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6da expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.840869:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660419584:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.840978:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.848041:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660419584:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845047.848086:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af2a0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6da expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.848129:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af2a0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6da expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.848143:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00000100:00100000:3.0:1788845047.848202:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641840896:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7430us (7663us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.848211:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1238 00000100:00100000:1.0:1788845047.852343:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.852496:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641841024 00000100:00100000:3.0:1788845047.852508:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641841024 00000100:00100000:3.0:1788845047.852569:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1641 00000100:00100000:3.0:1788845047.852580:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641841024:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.852608:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.852612:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845047.852657:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af245 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa686 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.852678:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.852683:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845047.852698:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6e1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.852724:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.852734:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.852766:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.852785:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 15 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.852801:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a10e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660419712:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845047.852878:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.856079:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a10e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660419712:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845047.856132:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.856150:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.856153:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.856158:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.856169:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845047.856221:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641841024:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3642us (3886us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.856230:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1641 00000100:00100000:1.0:1788845047.858857:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.859761:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641841152 00000100:00100000:3.0:1788845047.859774:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641841152 00010000:00010000:3.0:1788845047.859816:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 16 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.859846:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 563 00000100:00100000:3.0:1788845047.859855:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641841152:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845047.859869:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845047.859910:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 16 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.859927:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 16 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.859954:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af268 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6a2 expref: 16 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.859972:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.859978:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.859990:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.860000:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.860037:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.860051:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a10e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660419840:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845047.860319:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:1.0:1788845047.863394:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.863486:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641841280 00000100:00100000:0.0:1788845047.863496:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641841280 00000100:00100000:0.0:1788845047.863564:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1082 00000100:00100000:0.0:1788845047.863575:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641841280:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:0.0:1788845047.863812:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 244318 -> 244318 00000004:00010000:0.0:1788845047.863816:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845047 -> 1788845047 00000004:00010000:0.0:1788845047.863819:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845047 -> 1788845047 00000004:00010000:0.0:1788845047.863821:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845047 -> 1788845047 00000004:00010000:0.0:1788845047.863823:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:0.0:1788845047.863932:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641841280:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 357us (546us total) trans 4294967879 rc 0/0 00000100:00100000:0.0:1788845047.863940:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1082 00000100:00100000:3.0:1788845047.864503:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a10e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660419840:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845047.864525:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845047.864557:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641841152:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4703us (5710us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.864566:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 563 00000100:00100000:1.0:1788845047.868971:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.869124:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641841408 00000100:00100000:3.0:1788845047.869135:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641841408 00000100:00100000:3.0:1788845047.869226:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1642 00000100:00100000:3.0:1788845047.869237:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641841408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.869269:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.869289:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af2ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6e8 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.869326:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.869365:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a10e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660419968:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.869445:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22780 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.872251:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a10e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660419968:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845047.872295:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:2.0:1788845047.872315:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af2ae lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e8 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.872341:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.872344:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845047.872349:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af2ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e8 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.872364:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e800, rc 0) 00000100:00100000:2.0:1788845047.872428:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641841408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3192us (3464us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.872437:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1642 00000100:00100000:0.0:1788845047.875600:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.875930:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641841536 00000100:00100000:1.0:1788845047.875941:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641841536 00000100:00100000:1.0:1788845047.876002:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1643 00000100:00100000:1.0:1788845047.876013:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aaa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641841536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.876036:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.876056:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af2b5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6ef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.876118:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.876162:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660420096:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845047.876298:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845047.881818:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660420096:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845047.881851:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:1.0:1788845047.881870:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af2b5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6ef expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.881991:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845047.881995:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845047.882001:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af2b5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6ef expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845047.882017:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:1.0:1788845047.882214:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aaa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641841536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6202us (6635us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845047.882224:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1643 00000100:00100000:1.0:1788845047.885404:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.885556:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641841664 00000100:00100000:3.0:1788845047.885567:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641841664 00000100:00100000:3.0:1788845047.885670:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1644 00000100:00100000:3.0:1788845047.885681:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641841664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.885702:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.885722:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af2bc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6f6 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.885760:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.885800:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a10000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660420224:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.885905:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.888251:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a10000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660420224:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845047.888292:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:3.0:1788845047.888312:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af2bc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6f6 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.888333:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.888336:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.888342:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af2bc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6f6 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.888358:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845047.888417:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641841664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2737us (3022us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.888426:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1644 00000100:00100000:1.0:1788845047.896771:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.896822:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641841792 00000100:00100000:2.0:1788845047.896831:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641841792 00000100:00100000:2.0:1788845047.896879:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1645 00000100:00100000:2.0:1788845047.896908:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641841792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.896929:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.896949:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af2c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa6fd expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.896986:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.897026:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660420352:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.897092:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.899239:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660420352:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845047.899262:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:2.0:1788845047.899280:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af2c3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6fd expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.899301:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.899303:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845047.899306:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af2c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6fd expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.899318:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845047.899363:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2367:x1875739641841792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2459us (2601us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.899368:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1645 00000100:00100000:1.0:1788845047.903142:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.903294:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641841920 00000100:00100000:2.0:1788845047.903305:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641841920 00000100:00100000:2.0:1788845047.903362:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1646 00000100:00100000:2.0:1788845047.903373:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641841920:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.903394:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.903415:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af2ca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa704 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.903455:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.903494:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660420480:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.903572:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.909437:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660420480:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845047.909464:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:2.0:1788845047.909483:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af2ca lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa704 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.909504:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.909507:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845047.909513:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af2ca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa704 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.909527:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845047.909579:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641841920:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6208us (6447us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.909597:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1646 00000100:00100000:1.0:1788845047.912490:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.912626:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641842048 00000100:00100000:2.0:1788845047.912637:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641842048 00000100:00100000:2.0:1788845047.912696:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1647 00000100:00100000:2.0:1788845047.912706:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641842048:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.912726:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.912746:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af2d1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa70b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.912784:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.912824:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660420608:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.912918:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.915025:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660420608:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845047.915043:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:2.0:1788845047.915056:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af2d1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa70b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.915072:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845047.915074:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845047.915078:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af2d1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa70b expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.915088:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845047.915138:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641842048:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2433us (2655us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.915145:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1647 00000100:00100000:1.0:1788845047.918059:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.918354:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641842176 00000100:00100000:3.0:1788845047.918365:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641842176 00000100:00100000:3.0:1788845047.918550:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1648 00000100:00100000:3.0:1788845047.918563:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641842176:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.918595:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.918616:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af2d8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa712 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.918651:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.918690:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a10000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660420736:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.918812:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.925272:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a10000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660420736:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845047.925297:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:3.0:1788845047.925317:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af2d8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa712 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.925338:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.925341:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.925346:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af2d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa712 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.925361:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845047.925432:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641842176:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6872us (7381us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.925441:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1648 00000100:00100000:1.0:1788845047.928877:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.929008:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641842304 00000100:00100000:3.0:1788845047.929018:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641842304 00000100:00100000:3.0:1788845047.929081:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1649 00000100:00100000:3.0:1788845047.929091:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641842304:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.929121:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.929141:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af2df lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa719 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.929178:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.929219:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a10000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660420864:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.929302:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.932477:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a10000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660420864:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845047.932504:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:3.0:1788845047.932525:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af2df lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa719 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.932547:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845047.932550:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845047.932556:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af2df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa719 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.932571:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:3.0:1788845047.932646:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641842304:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3554us (3776us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.932654:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1649 00000100:00100000:1.0:1788845047.936479:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.936858:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641842432 00000100:00100000:0.0:1788845047.936868:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641842432 00000100:00100000:1.0:1788845047.937396:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1239 00000100:00100000:1.0:1788845047.937409:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641842432:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.937429:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.937448:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287af2e6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa720 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.937512:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02361880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660420992:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845047.937603:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.940564:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02361880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660420992:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845047.940619:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287af2e6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa720 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.940644:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dc0aea3f/0xb2187301287af2e6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa720 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.940657:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1800, rc 0) 00000100:00100000:3.0:1788845047.940762:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641842432:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3355us (4300us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.940770:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1239 00000100:00100000:1.0:1788845047.944545:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.944661:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641842560 00000100:00100000:0.0:1788845047.944672:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641842560 00000100:00100000:0.0:1788845047.944761:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1240 00000100:00100000:0.0:1788845047.944773:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641842560:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845047.944796:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845047.944813:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287af2ed lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa727 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845047.944877:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a04000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660421120:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845047.944990:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c183c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.947627:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a04000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660421120:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.947665:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287af2ed lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa727 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.947690:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001496a08f/0xb2187301287af2ed lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa727 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.947703:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007c00, rc 0) 00000100:00100000:0.0:1788845047.947799:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641842560:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3027us (3337us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.947808:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1240 00000100:00100000:1.0:1788845047.950810:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.951003:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641842688 00000100:00100000:0.0:1788845047.951014:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641842688 00002000:00100000:0.0:1788845047.951062:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02363100 x1875739641842688/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.951073:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02363100 with x1875739641842688 00000100:00100000:0.0:1788845047.951320:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1083 00000100:00100000:0.0:1788845047.951333:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641842688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.951663:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02363100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641842688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 331us (860us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845047.951669:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1083 00002000:00100000:0.0:1788845047.951677:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f02363100 x1875739641842688/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.951684:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f02363100 with x1875739641842688 00000100:00100000:1.0:1788845047.954978:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.955066:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641842816 00000100:00100000:2.0:1788845047.955077:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641842816 00000100:00100000:2.0:1788845047.955251:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1241 00000100:00100000:2.0:1788845047.955273:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641842816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.955292:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.955309:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af2f4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa72e expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.955365:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660421248:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.955483:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.959697:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660421248:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.959739:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af2f4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa72e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.959764:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af2f4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa72e expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.959779:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845047.959878:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641842816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4608us (4909us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.959912:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1241 00000100:00100000:1.0:1788845047.963391:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845047.963569:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641842944 00000100:00100000:0.0:1788845047.963578:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641842944 00002000:00100000:0.0:1788845047.963635:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999c00 x1875739641842944/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.963646:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999c00 with x1875739641842944 00000100:00100000:0.0:1788845047.963717:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1084 00000100:00100000:0.0:1788845047.963727:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641842944:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845047.964007:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641842944:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (627us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845047.964014:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1084 00002000:00100000:0.0:1788845047.964021:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999c00 x1875739641842944/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845047.964035:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999c00 with x1875739641842944 00000100:00100000:3.0:1788845047.967058:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845047.967180:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641843072 00000100:00100000:2.0:1788845047.967190:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641843072 00000100:00100000:2.0:1788845047.967245:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1242 00000100:00100000:2.0:1788845047.967256:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641843072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845047.967275:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845047.967293:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af2fb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa735 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845047.967351:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660421376:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845047.967426:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845047.970628:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660421376:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845047.970669:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af2fb lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa735 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.970694:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af2fb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa735 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845047.970707:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845047.970765:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641843072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3511us (3719us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845047.970773:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1242 00000100:00100000:1.0:1788845047.973602:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.973693:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641843200 00000100:00100000:3.0:1788845047.973703:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641843200 00002000:00100000:3.0:1788845047.973757:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999f80 x1875739641843200/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.973768:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999f80 with x1875739641843200 00000100:00100000:3.0:1788845047.973811:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1085 00000100:00100000:3.0:1788845047.973821:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641843200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.974102:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641843200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (518us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.974124:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1085 00002000:00100000:3.0:1788845047.974132:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999f80 x1875739641843200/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.974139:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999f80 with x1875739641843200 00000100:00100000:3.0:1788845047.977768:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.977865:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641843328 00000100:00100000:1.0:1788845047.977875:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641843328 00000100:00100000:1.0:1788845047.977966:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1243 00000100:00100000:1.0:1788845047.977976:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641843328:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845047.977995:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845047.978014:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af302 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa73c expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845047.978073:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03999500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660421504:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845047.978164:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845047.984919:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03999500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660421504:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845047.984960:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af302 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa73c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.984984:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af302 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa73c expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845047.984997:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:3.0:1788845047.985055:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641843328:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7080us (7295us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845047.985063:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1243 00000100:00100000:3.0:1788845047.988088:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845047.988238:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641843456 00000100:00100000:1.0:1788845047.988248:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641843456 00002000:00100000:1.0:1788845047.988306:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a12a00 x1875739641843456/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.988317:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a12a00 with x1875739641843456 00000100:00100000:1.0:1788845047.988354:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1086 00000100:00100000:1.0:1788845047.988364:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a12a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641843456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845047.988642:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a12a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641843456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 279us (564us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845047.988650:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1086 00002000:00100000:1.0:1788845047.988657:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a12a00 x1875739641843456/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845047.988666:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a12a00 with x1875739641843456 00000100:00100000:0.0:1788845047.992248:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.992333:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641843584 00000100:00100000:3.0:1788845047.992342:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641843584 00000100:00100000:3.0:1788845047.992400:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1244 00000100:00100000:3.0:1788845047.992411:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a04000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641843584:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845047.992430:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845047.992449:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287af309 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa743 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845047.992515:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660421632:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845047.992610:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845047.995102:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660421632:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845047.995140:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287af309 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa743 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.995167:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093afb5ab/0xb2187301287af309 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa743 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845047.995181:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:0.0:1788845047.995240:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a04000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641843584:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2831us (3001us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845047.995249:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1244 00000100:00100000:1.0:1788845047.998749:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845047.998843:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641843712 00000100:00100000:3.0:1788845047.998854:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641843712 00002000:00100000:3.0:1788845047.998946:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999180 x1875739641843712/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/0 e 0 to 0 dl 1788845058 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.998957:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999180 with x1875739641843712 00000100:00100000:3.0:1788845047.998981:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1087 00000100:00100000:3.0:1788845047.998992:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641843712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845047.999297:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641843712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 305us (557us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845047.999306:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1087 00002000:00100000:3.0:1788845047.999313:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03999180 x1875739641843712/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:153/0 lens 488/440 e 0 to 0 dl 1788845058 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845047.999320:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03999180 with x1875739641843712 00000100:00100000:3.0:1788845048.003140:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.003768:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641843840 00000100:00100000:2.0:1788845048.003778:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641843840 00000100:00100000:2.0:1788845048.003914:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1245 00000100:00100000:2.0:1788845048.003927:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641843840:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.003947:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.003966:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af310 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa74a expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.004028:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660421760:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.004126:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.006540:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660421760:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845048.006580:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af310 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa74a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.006616:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af310 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa74a expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.006631:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845048.006689:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641843840:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2764us (3559us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.006698:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1245 00000100:00100000:1.0:1788845048.010313:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.010409:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641843968 00000100:00100000:3.0:1788845048.010418:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641843968 00002000:00100000:3.0:1788845048.010480:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399b480 x1875739641843968/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.010491:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399b480 with x1875739641843968 00000100:00100000:3.0:1788845048.010530:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1088 00000100:00100000:3.0:1788845048.010541:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641843968:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.010801:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641843968:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 260us (497us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.010808:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1088 00002000:00100000:3.0:1788845048.010816:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399b480 x1875739641843968/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.010823:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399b480 with x1875739641843968 00000100:00100000:3.0:1788845048.013983:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.014075:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641844096 00000100:00100000:2.0:1788845048.014084:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641844096 00000100:00100000:2.0:1788845048.014147:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1246 00000100:00100000:2.0:1788845048.014157:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a13800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641844096:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.014175:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.014193:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af317 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa751 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.014254:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660421888:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.014343:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.016756:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660421888:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845048.016795:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af317 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa751 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.016820:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af317 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa751 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.016834:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845048.016951:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a13800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641844096:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2794us (2976us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.016960:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1246 00000100:00100000:1.0:1788845048.019982:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.020077:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641844224 00000100:00100000:3.0:1788845048.020087:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641844224 00002000:00100000:3.0:1788845048.020149:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399a680 x1875739641844224/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.020161:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399a680 with x1875739641844224 00000100:00100000:3.0:1788845048.020202:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1089 00000100:00100000:3.0:1788845048.020211:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641844224:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.020476:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641844224:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 265us (503us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.020485:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1089 00002000:00100000:3.0:1788845048.020492:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399a680 x1875739641844224/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.020506:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399a680 with x1875739641844224 00000100:00100000:3.0:1788845048.024208:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.024313:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641844352 00000100:00100000:1.0:1788845048.024322:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641844352 00000100:00100000:1.0:1788845048.024378:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1247 00000100:00100000:1.0:1788845048.024388:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a12d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641844352:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.024407:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.024427:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af31e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa758 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.024497:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660422016:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.024580:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.027276:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660422016:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845048.027321:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af31e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa758 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.027346:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af31e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa758 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.027359:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:3.0:1788845048.027424:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a12d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641844352:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3036us (3223us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.027433:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1247 00000100:00100000:3.0:1788845048.030570:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.030727:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641844480 00000100:00100000:1.0:1788845048.030738:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641844480 00002000:00100000:1.0:1788845048.030785:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a10e00 x1875739641844480/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845048.030796:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a10e00 with x1875739641844480 00000100:00100000:1.0:1788845048.030838:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1090 00000100:00100000:1.0:1788845048.030849:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641844480:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845048.031128:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641844480:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (566us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845048.031136:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1090 00002000:00100000:1.0:1788845048.031143:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a10e00 x1875739641844480/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845048.031150:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a10e00 with x1875739641844480 00000100:00100000:0.0:1788845048.035002:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.035221:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641844608 00000100:00100000:3.0:1788845048.035231:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641844608 00000100:00100000:3.0:1788845048.035385:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1248 00000100:00100000:3.0:1788845048.035395:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a05500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641844608:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.035415:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.035433:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287af325 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa75f expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.035506:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03446300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660422144:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.035603:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22600 going to sleep for 11 seconds 00000100:00100000:0.0:1788845048.037861:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03446300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660422144:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845048.037954:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287af325 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa75f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.037980:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603d8098/0xb2187301287af325 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa75f expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.037995:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:0.0:1788845048.038087:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a05500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641844608:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2693us (3094us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.038111:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1248 00000100:00100000:0.0:1788845048.041174:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.041477:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641844736 00000100:00100000:3.0:1788845048.041488:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641844736 00002000:00100000:3.0:1788845048.041536:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a05180 x1875739641844736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.041548:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a05180 with x1875739641844736 00000100:00100000:3.0:1788845048.041599:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1091 00000100:00100000:3.0:1788845048.041609:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a05180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641844736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.041870:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a05180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641844736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 261us (703us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.041877:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1091 00002000:00100000:3.0:1788845048.041907:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a05180 x1875739641844736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.041915:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a05180 with x1875739641844736 00000100:00100000:1.0:1788845048.045552:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.045704:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641844864 00000100:00100000:2.0:1788845048.045714:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641844864 00000100:00100000:2.0:1788845048.045773:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1249 00000100:00100000:2.0:1788845048.045783:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641844864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.045802:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.045819:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af32c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa766 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.045876:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660422272:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.045988:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.048736:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660422272:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845048.048776:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af32c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa766 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.048799:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af32c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa766 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.048812:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845048.048997:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641844864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3214us (3452us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.049006:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1249 00000100:00100000:1.0:1788845048.051234:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.051363:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641844992 00000100:00100000:3.0:1788845048.051374:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641844992 00002000:00100000:3.0:1788845048.051419:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399ad80 x1875739641844992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.051432:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399ad80 with x1875739641844992 00000100:00100000:3.0:1788845048.051464:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1092 00000100:00100000:3.0:1788845048.051472:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641844992:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.051715:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641844992:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (490us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.051721:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1092 00002000:00100000:3.0:1788845048.051726:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399ad80 x1875739641844992/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.051735:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399ad80 with x1875739641844992 00000100:00100000:1.0:1788845048.054283:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.054369:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641845120 00000100:00100000:2.0:1788845048.054379:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641845120 00000100:00100000:2.0:1788845048.054439:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1250 00000100:00100000:2.0:1788845048.054449:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641845120:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.054482:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.054501:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af333 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa76d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.054559:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660422400:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.054638:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.056549:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660422400:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845048.056602:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af333 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa76d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.056626:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cbb00a5/0xb2187301287af333 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa76d expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.056638:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845048.056721:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641845120:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2273us (2448us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.056729:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1250 00000100:00100000:1.0:1788845048.059170:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.059233:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641845248 00000100:00100000:3.0:1788845048.059244:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641845248 00002000:00100000:3.0:1788845048.059291:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998a80 x1875739641845248/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.059303:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998a80 with x1875739641845248 00000100:00100000:3.0:1788845048.059341:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1093 00000100:00100000:3.0:1788845048.059351:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641845248:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.059591:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641845248:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 232us (425us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.059600:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1093 00002000:00100000:3.0:1788845048.059607:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03998a80 x1875739641845248/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.059615:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03998a80 with x1875739641845248 00000100:00100000:1.0:1788845048.061605:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.061659:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641845376 00000100:00100000:3.0:1788845048.061666:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641845376 00000100:00100000:3.0:1788845048.061708:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1251 00000100:00100000:3.0:1788845048.061715:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641845376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.061732:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.061747:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af33a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa774 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.061791:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660422528:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.061854:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.063682:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660422528:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845048.063715:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af33a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa774 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.063734:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af33a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa774 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.063745:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:3.0:1788845048.063792:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641845376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2076us (2206us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.063801:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1251 00000100:00100000:1.0:1788845048.065810:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.065851:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641845504 00000100:00100000:3.0:1788845048.065861:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641845504 00002000:00100000:3.0:1788845048.065929:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399aa00 x1875739641845504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.065942:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399aa00 with x1875739641845504 00000100:00100000:0.0:1788845048.065984:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1094 00000100:00100000:0.0:1788845048.065997:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641845504:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.066436:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641845504:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 437us (628us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.066443:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1094 00002000:00100000:0.0:1788845048.066451:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399aa00 x1875739641845504/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.066459:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399aa00 with x1875739641845504 00000100:00100000:1.0:1788845048.068539:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.068621:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641845632 00000100:00100000:2.0:1788845048.068629:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641845632 00000100:00100000:2.0:1788845048.068673:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1650 00000100:00100000:2.0:1788845048.068681:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641845632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.068703:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.068721:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af341 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa77b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.068753:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.068786:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660422656:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.068856:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.070817:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660422656:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845048.070848:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:2.0:1788845048.070866:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af341 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa77b expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.070915:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.070918:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.070924:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af341 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa77b expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.070938:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845048.070997:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641845632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2315us (2468us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.071006:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1650 00000100:00100000:1.0:1788845048.074285:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.074386:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641845760 00000100:00100000:2.0:1788845048.074394:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641845760 00000100:00100000:2.0:1788845048.074450:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1651 00000100:00100000:2.0:1788845048.074460:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641845760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.074539:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.074561:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af348 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa782 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.074608:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.074652:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660422784:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.074759:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.076686:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660422784:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845048.076713:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:2.0:1788845048.076732:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af348 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa782 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.076754:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.076757:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.076762:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af348 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa782 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.076777:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845048.076866:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641845760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2406us (2589us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.076874:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1651 00000100:00100000:1.0:1788845048.079979:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.080033:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641845888 00000100:00100000:3.0:1788845048.080042:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641845888 00000100:00100000:3.0:1788845048.080091:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1652 00000100:00100000:3.0:1788845048.080111:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03999500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641845888:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.080266:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.080287:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af34f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa789 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.080319:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.080352:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660422912:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.080421:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.082019:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660422912:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845048.082040:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:3.0:1788845048.082054:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af34f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa789 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.082069:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.082071:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845048.082074:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af34f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa789 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.082087:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:2.0:1788845048.082366:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03999500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2370:x1875739641845888:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2264us (2392us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.082373:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1652 00000100:00100000:1.0:1788845048.084354:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.084475:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641846016 00000100:00100000:3.0:1788845048.084482:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641846016 00000100:00100000:3.0:1788845048.084527:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1653 00000100:00100000:3.0:1788845048.084537:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641846016:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.084557:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.084575:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af356 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa790 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.084625:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.084666:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660423040:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.084768:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.086675:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660423040:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845048.086699:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:3.0:1788845048.086715:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af356 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa790 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.086733:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.086736:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845048.086740:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af356 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa790 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.086754:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845048.086803:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2369:x1875739641846016:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2267us (2455us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.086811:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1653 00000100:00100000:1.0:1788845048.088967:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.089028:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641846144 00000100:00100000:2.0:1788845048.089037:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641846144 00000100:00100000:1.0:1788845048.089175:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1654 00000100:00100000:1.0:1788845048.089251:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641846144:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.089313:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.089333:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af35d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa797 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.089364:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.089396:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01967b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660423168:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.089522:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.091549:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01967b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660423168:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845048.091575:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 244318 00010000:00010000:2.0:1788845048.091660:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af35d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa797 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.091680:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.091683:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.091688:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af35d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa797 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.091708:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:2.0:1788845048.091800:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:2368:x1875739641846144:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2553us (2838us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.091809:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1654 00000100:00100000:0.0:1788845048.098968:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.099220:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641846272 00000100:00100000:1.0:1788845048.099231:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641846272 00000100:00100000:1.0:1788845048.099294:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1655 00000100:00100000:1.0:1788845048.099304:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a07480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+15:14891:x1875739641846272:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.099328:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.099351:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa79e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.099384:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.099395:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.099433:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.099450:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 15 pid: 5839 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.099650:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01966d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660423296:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845048.099736:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.104945:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.105125:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641846400 00000100:00100000:3.0:1788845048.105136:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641846400 00010000:00010000:3.0:1788845048.105397:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 16 pid: 5839 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.105436:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 564 00000100:00100000:3.0:1788845048.105447:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641846400:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845048.105469:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845048.105494:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 16 pid: 5839 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.105511:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 16 pid: 5839 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.105529:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.105534:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.105542:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.105549:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.105578:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.105600:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660423424:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845048.105665:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22a80 going to sleep for 11 seconds 00010000:00010000:2.0:1788845048.106770:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af2a7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa6e1 expref: 16 pid: 5839 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.106791:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01966d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660423296:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845048.106817:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.106831:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.106835:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.106838:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1e00, rc 0) 00000100:00100000:2.0:1788845048.106939:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a07480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:14891:x1875739641846272:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7637us (7983us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.106945:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1655 00000100:00100000:1.0:1788845048.110457:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.110594:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641846528 00000100:00100000:3.0:1788845048.110603:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641846528 00002000:00100000:3.0:1788845048.110651:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964000 x1875739641846528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.110662:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964000 with x1875739641846528 00000100:00100000:3.0:1788845048.110696:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1095 00000100:00100000:3.0:1788845048.110706:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641846528:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.110981:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641846528:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 275us (531us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.110988:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1095 00002000:00100000:3.0:1788845048.110996:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964000 x1875739641846528/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.111004:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964000 with x1875739641846528 00000100:00100000:0.0:1788845048.112858:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03444a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660423424:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845048.112901:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845048.112959:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641846400:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7514us (8022us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.112970:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 564 00000100:00100000:1.0:1788845048.116792:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.117002:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641846656 00000100:00100000:2.0:1788845048.117013:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641846656 00002000:00100000:2.0:1788845048.117061:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966680 x1875739641846656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.117073:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966680 with x1875739641846656 00000100:00100000:2.0:1788845048.117128:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1096 00000100:00100000:2.0:1788845048.117138:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641846656:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.117405:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641846656:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (624us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.117413:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1096 00002000:00100000:2.0:1788845048.117420:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966680 x1875739641846656/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.117428:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966680 with x1875739641846656 00000100:00100000:1.0:1788845048.120529:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.120624:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641846784 00000100:00100000:3.0:1788845048.120633:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641846784 00002000:00100000:3.0:1788845048.120684:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964a80 x1875739641846784/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.120694:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964a80 with x1875739641846784 00000100:00100000:3.0:1788845048.120730:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1097 00000100:00100000:3.0:1788845048.120740:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641846784:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.121046:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641846784:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 306us (526us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.121054:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1097 00002000:00100000:3.0:1788845048.121061:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01964a80 x1875739641846784/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.121069:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01964a80 with x1875739641846784 00000100:00100000:1.0:1788845048.125139:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.125385:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641846912 00000100:00100000:2.0:1788845048.125396:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641846912 00002000:00100000:2.0:1788845048.125448:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966a00 x1875739641846912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.125459:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966a00 with x1875739641846912 00000100:00100000:2.0:1788845048.125623:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1098 00000100:00100000:2.0:1788845048.125634:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641846912:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.125923:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641846912:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 291us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.125928:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1098 00002000:00100000:2.0:1788845048.125934:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966a00 x1875739641846912/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.125939:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966a00 with x1875739641846912 00000100:00100000:1.0:1788845048.128502:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.128571:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641847040 00000100:00100000:3.0:1788845048.128578:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641847040 00002000:00100000:3.0:1788845048.128635:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967800 x1875739641847040/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.128645:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967800 with x1875739641847040 00000100:00100000:3.0:1788845048.128681:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1099 00000100:00100000:3.0:1788845048.128690:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641847040:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.128911:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641847040:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 221us (416us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.128922:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1099 00002000:00100000:3.0:1788845048.128929:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967800 x1875739641847040/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.128936:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967800 with x1875739641847040 00000100:00100000:1.0:1788845048.132606:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.132723:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641847168 00000100:00100000:0.0:1788845048.132733:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641847168 00002000:00100000:0.0:1788845048.132779:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966d80 x1875739641847168/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.132791:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966d80 with x1875739641847168 00000100:00100000:0.0:1788845048.132917:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1100 00000100:00100000:0.0:1788845048.132927:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641847168:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.133190:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641847168:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (594us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.133199:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1100 00002000:00100000:0.0:1788845048.133207:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01966d80 x1875739641847168/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.133214:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01966d80 with x1875739641847168 00000100:00100000:1.0:1788845048.136855:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.136976:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641847296 00000100:00100000:0.0:1788845048.136985:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641847296 00002000:00100000:0.0:1788845048.137032:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967b80 x1875739641847296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.137042:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967b80 with x1875739641847296 00000100:00100000:0.0:1788845048.137079:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1101 00000100:00100000:0.0:1788845048.137089:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641847296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.137331:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2369:x1875739641847296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 238us (480us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.137336:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1101 00002000:00100000:0.0:1788845048.137341:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01967b80 x1875739641847296/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.137346:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01967b80 with x1875739641847296 00000100:00100000:1.0:1788845048.140880:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.141020:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641847424 00000100:00100000:2.0:1788845048.141030:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641847424 00002000:00100000:2.0:1788845048.141077:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48380 x1875739641847424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.141088:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48380 with x1875739641847424 00000100:00100000:2.0:1788845048.141115:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1102 00000100:00100000:2.0:1788845048.141125:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641847424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.141372:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641847424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 248us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.141380:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1102 00002000:00100000:2.0:1788845048.141387:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48380 x1875739641847424/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.141396:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48380 with x1875739641847424 00000100:00100000:1.0:1788845048.145313:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.145598:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641847552 00000100:00100000:0.0:1788845048.145609:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641847552 00002000:00100000:0.0:1788845048.145655:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4a300 x1875739641847552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.145664:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4a300 with x1875739641847552 00000100:00100000:0.0:1788845048.145702:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1103 00000100:00100000:0.0:1788845048.145711:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641847552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.145990:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2367:x1875739641847552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (686us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.145998:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1103 00002000:00100000:0.0:1788845048.146005:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4a300 x1875739641847552/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.146012:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4a300 with x1875739641847552 00000100:00100000:1.0:1788845048.149667:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.149769:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641847680 00000100:00100000:2.0:1788845048.149779:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641847680 00002000:00100000:2.0:1788845048.149828:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48000 x1875739641847680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.149838:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48000 with x1875739641847680 00000100:00100000:2.0:1788845048.149879:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1104 00000100:00100000:2.0:1788845048.149926:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641847680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.150167:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2368:x1875739641847680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (508us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.150175:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1104 00002000:00100000:2.0:1788845048.150182:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48000 x1875739641847680/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.150190:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48000 with x1875739641847680 00000100:00100000:1.0:1788845048.155054:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.155264:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641847808 00000100:00100000:0.0:1788845048.155282:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641847808 00002000:00100000:0.0:1788845048.155439:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4a680 x1875739641847808/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.155451:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4a680 with x1875739641847808 00000100:00100000:0.0:1788845048.155598:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1105 00000100:00100000:0.0:1788845048.155608:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+16:2370:x1875739641847808:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:2.0:1788845048.160211:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 244318 -> 528384 00000004:00010000:2.0:1788845048.160222:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845047 -> 1788845048 00000004:00010000:2.0:1788845048.160225:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845047 -> 1788845047 00000004:00010000:2.0:1788845048.160228:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845047 -> 1788845048 00000004:00010000:2.0:1788845048.160230:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:2.0:1788845048.161809:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2370:x1875739641847808:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 6191us (6754us total) trans 4294967880 rc 0/0 00000100:00100000:2.0:1788845048.161822:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1105 00002000:00100000:2.0:1788845048.161835:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da4a680 x1875739641847808/t4294967880(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/448 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.161845:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da4a680 with x1875739641847808 00000100:00100000:3.0:1788845048.165518:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.165647:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641847936 00000100:00100000:2.0:1788845048.165657:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641847936 00000100:00100000:2.0:1788845048.165722:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1656 00000100:00100000:2.0:1788845048.165732:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+18:2368:x1875739641847936:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845048.207053:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967879 is committed 00000001:00080000:1.0:1788845048.207072:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967878 is committed 00000001:00080000:1.0:1788845048.207117:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967880 is committed 00000100:00100000:3.0:1788845048.207342:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641847936:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 41596us (42079us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.207355:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1656 00000100:00100000:3.0:1788845048.212298:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.212557:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.212667:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641848064 00000100:00100000:2.0:1788845048.212677:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641848064 00000100:00100000:2.0:1788845048.212740:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1657 00000100:00100000:2.0:1788845048.212750:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641848064:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.212773:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.212794:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af36b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7a5 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.212833:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.212874:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660423552:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.212986:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845048.213609:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641848192 00000100:00100000:0.0:1788845048.213615:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641848192 00000100:00100000:0.0:1788845048.213666:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1252 00000100:00100000:0.0:1788845048.213674:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641848192:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.213685:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.213696:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287af372 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7ac expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.213736:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a06680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660423680:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845048.213774:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18b40 going to sleep for 11 seconds 00000004:00010000:2.0:1788845048.218129:0:5840:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845047 -> 1788845048 00000100:00100000:2.0:1788845048.218163:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660423552:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845048.218199:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:2.0:1788845048.218220:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af36b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7a5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.218239:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.218242:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.218247:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af36b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7a5 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.218263:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845048.218646:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03446680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641848064:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5895us (6086us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.218656:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1657 00002000:00010000:0.0:1788845048.226973:0:7983:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime: 1788845047 -> 1788845048 00002000:00010000:0.0:1788845048.226983:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845047 -> 1788845048 00002000:00010000:0.0:1788845048.226986:0:7983:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845047 -> 1788845048 00000100:00100000:0.0:1788845048.227013:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a06680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660423680:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845048.227054:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287af372 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7ac expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.227077:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a604fb93/0xb2187301287af372 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7ac expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.227092:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845048.228777:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641848192:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 15094us (16479us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.228790:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1252 00000100:00100000:3.0:1788845048.235662:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.235705:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.235793:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641848320 00000100:00100000:1.0:1788845048.235803:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641848320 00000100:00100000:1.0:1788845048.235865:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1658 00000100:00100000:1.0:1788845048.235876:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641848320:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.235929:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.235953:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af379 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7b3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.235993:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.236036:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660423808:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.236128:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.238354:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641848448 00000100:00100000:2.0:1788845048.238370:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641848448 00000100:00100000:2.0:1788845048.238446:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1253 00000100:00100000:2.0:1788845048.238466:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641848448:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.238484:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.238503:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af380 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7ba expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.238560:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660423936:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.238658:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.242235:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660423808:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845048.242262:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:1.0:1788845048.242292:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af379 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7b3 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.242313:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845048.242316:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845048.242321:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af379 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7b3 expref: 15 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.242348:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1000, rc 0) 00000100:00100000:1.0:1788845048.242412:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641848320:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6538us (6705us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.242420:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1658 00000100:00100000:0.0:1788845048.251222:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03ae8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660423936:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845048.251277:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af380 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7ba expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.251303:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005055cb9b/0xb2187301287af380 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7ba expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.251317:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:0.0:1788845048.251409:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641848448:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 12953us (15755us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.251418:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1253 00000100:00100000:2.0:1788845048.258223:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.258349:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641848576 00000100:00100000:3.0:1788845048.258359:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641848576 00000100:00100000:3.0:1788845048.258427:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1659 00000100:00100000:3.0:1788845048.258438:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641848576:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.258458:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.258488:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af387 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7c1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.258524:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.258565:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660424064:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.260559:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.262420:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.262594:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641848704 00000100:00100000:1.0:1788845048.262605:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641848704 00000100:00100000:1.0:1788845048.262677:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1254 00000100:00100000:1.0:1788845048.262687:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641848704:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.262705:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.262723:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287af38e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7c8 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.262782:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660424192:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.262850:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.266131:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660424192:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845048.266173:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287af38e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7c8 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.266211:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c60cfc9/0xb2187301287af38e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7c8 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.266225:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1000, rc 0) 00000100:00100000:1.0:1788845048.266290:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641848704:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3604us (3878us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.266299:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1254 00000100:00100000:3.0:1788845048.269831:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660424064:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845048.269859:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:3.0:1788845048.269916:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af387 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7c1 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.269939:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.269943:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845048.269949:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fe00/0xb2187301287af387 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7c1 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.269965:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:3.0:1788845048.270016:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641848576:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11580us (11803us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.270025:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1659 00000100:00100000:0.0:1788845048.277168:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.277433:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641848832 00000100:00100000:3.0:1788845048.277443:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641848832 00000100:00100000:3.0:1788845048.277505:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1660 00000100:00100000:3.0:1788845048.277515:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a06680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:14891:x1875739641848832:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.277536:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.277555:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7cf expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.277592:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.277603:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7cf expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.277641:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 10 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.277660:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 10 pid: 5841 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.277675:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660424320:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845048.277755:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22540 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.288174:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660424320:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845048.288219:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7cf expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.288241:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.288244:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845048.288248:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7cf expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.288814:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845048.288923:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a06680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641848832:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 11408us (11764us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.288933:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1660 00000100:00100000:2.0:1788845048.295536:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.295835:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641848960 00000100:00100000:1.0:1788845048.295856:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641848960 00010000:00010000:1.0:1788845048.295933:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 11 pid: 5841 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.295968:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 565 00000100:00100000:1.0:1788845048.295978:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008d12050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641848960:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845048.295991:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845048.296016:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 11 pid: 5841 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.296034:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 11 pid: 5841 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.296060:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1e00/0xb2187301287af364 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa79e expref: 11 pid: 5841 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.296075:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845048.296081:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7cf expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.296093:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7cf expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.296102:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7cf expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.296140:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7cf expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.296154:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660424448:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845048.296225:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8f900 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.303394:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.303601:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da48a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660424448:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845048.303628:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845048.303690:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008d12050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641848960:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 7712us (8162us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845048.303701:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 565 00000100:00100000:3.0:1788845048.305001:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641849088 00000100:00100000:3.0:1788845048.305013:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641849088 00002000:00100000:3.0:1788845048.305065:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03aeaa00 x1875739641849088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.305080:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aeaa00 with x1875739641849088 00000100:00100000:3.0:1788845048.305127:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1106 00000100:00100000:3.0:1788845048.305137:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aeaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641849088:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.306870:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aeaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641849088:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 1726us (3477us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845048.306907:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1106 00002000:00100000:3.0:1788845048.306917:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03aeaa00 x1875739641849088/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.306927:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aeaa00 with x1875739641849088 00000100:00100000:2.0:1788845048.313675:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.314567:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641849216 00000100:00100000:0.0:1788845048.314590:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641849216 00002000:00100000:0.0:1788845048.314641:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae9180 x1875739641849216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.314652:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae9180 with x1875739641849216 00000100:00100000:0.0:1788845048.314693:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1107 00000100:00100000:0.0:1788845048.314703:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641849216:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.315013:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641849216:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 311us (1348us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845048.315022:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1107 00002000:00100000:0.0:1788845048.315028:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae9180 x1875739641849216/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.315036:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae9180 with x1875739641849216 00000100:00100000:2.0:1788845048.328732:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.328833:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641849344 00000100:00100000:0.0:1788845048.328844:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641849344 00002000:00100000:0.0:1788845048.328933:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03aea680 x1875739641849344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.328945:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aea680 with x1875739641849344 00000100:00100000:0.0:1788845048.328982:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1108 00000100:00100000:0.0:1788845048.328992:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641849344:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.329255:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641849344:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 263us (531us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845048.329263:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1108 00002000:00100000:0.0:1788845048.330027:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03aea680 x1875739641849344/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.330042:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03aea680 with x1875739641849344 00000100:00100000:3.0:1788845048.334475:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.334698:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641849472 00000100:00100000:0.0:1788845048.334708:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641849472 00002000:00100000:0.0:1788845048.334758:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445500 x1875739641849472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.334769:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445500 with x1875739641849472 00000100:00100000:0.0:1788845048.334809:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1109 00000100:00100000:0.0:1788845048.334819:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641849472:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.335142:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641849472:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 324us (687us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845048.335152:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1109 00002000:00100000:0.0:1788845048.335159:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03445500 x1875739641849472/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.335166:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03445500 with x1875739641849472 00000100:00100000:2.0:1788845048.357539:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.357618:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641849600 00000100:00100000:3.0:1788845048.357637:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641849600 00002000:00100000:3.0:1788845048.357691:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae8000 x1875739641849600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.357703:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae8000 with x1875739641849600 00000100:00100000:3.0:1788845048.357741:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1110 00000100:00100000:3.0:1788845048.357751:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641849600:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.358000:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641849600:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 250us (467us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845048.358007:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1110 00002000:00100000:3.0:1788845048.358014:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae8000 x1875739641849600/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.358021:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae8000 with x1875739641849600 00000100:00100000:3.0:1788845048.367931:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.368035:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641849728 00000100:00100000:0.0:1788845048.368046:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641849728 00002000:00100000:0.0:1788845048.368344:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03447480 x1875739641849728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.368355:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03447480 with x1875739641849728 00000100:00100000:0.0:1788845048.368389:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1111 00000100:00100000:0.0:1788845048.368400:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03447480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641849728:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.368746:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03447480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641849728:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 347us (864us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845048.368757:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1111 00002000:00100000:0.0:1788845048.368764:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03447480 x1875739641849728/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.368772:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03447480 with x1875739641849728 00000100:00100000:1.0:1788845048.375014:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.375318:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641849856 00000100:00100000:3.0:1788845048.375329:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641849856 00002000:00100000:3.0:1788845048.375380:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48a80 x1875739641849856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.375393:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48a80 with x1875739641849856 00000100:00100000:3.0:1788845048.375500:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1112 00000100:00100000:3.0:1788845048.375510:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641849856:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.375762:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641849856:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (757us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845048.375771:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1112 00002000:00100000:3.0:1788845048.375779:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a02da48a80 x1875739641849856/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.375787:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a02da48a80 with x1875739641849856 00000100:00100000:2.0:1788845048.385790:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.385933:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641849984 00000100:00100000:0.0:1788845048.385943:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641849984 00002000:00100000:0.0:1788845048.386008:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae9500 x1875739641849984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.386019:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae9500 with x1875739641849984 00000100:00100000:0.0:1788845048.386046:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1113 00000100:00100000:0.0:1788845048.386056:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641849984:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.386556:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641849984:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 499us (774us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788845048.386564:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1113 00002000:00100000:0.0:1788845048.386572:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03ae9500 x1875739641849984/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.386590:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03ae9500 with x1875739641849984 00000100:00100000:2.0:1788845048.398053:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.398560:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641850112 00000100:00100000:1.0:1788845048.398571:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641850112 00000100:00100000:1.0:1788845048.398681:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1255 00000100:00100000:1.0:1788845048.398691:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641850112:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.398709:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.398727:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287af39c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7d6 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.398786:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660424576:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.398880:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.402048:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660424576:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845048.402088:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287af39c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7d6 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.402115:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287af39c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7d6 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.402129:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0e00, rc 0) 00000100:00100000:3.0:1788845048.402185:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641850112:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3495us (4140us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.402208:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1255 00010000:00010000:1.0:1788845048.402499:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845048.405990:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.406560:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641850240 00000100:00100000:1.0:1788845048.406571:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641850240 00000100:00100000:1.0:1788845048.406688:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1256 00000100:00100000:1.0:1788845048.406700:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641850240:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.406717:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.406734:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287af3a3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7dd expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.406793:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660424704:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.406876:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.410418:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a02da4aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660424704:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845048.410453:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287af3a3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7dd expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.410478:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287af3a3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7dd expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.410498:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:1.0:1788845048.410550:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641850240:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3853us (4567us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.410557:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1256 00000100:00100000:3.0:1788845048.416045:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.416376:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641850368 00000100:00100000:0.0:1788845048.416385:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641850368 00000100:00100000:0.0:1788845048.416441:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1661 00000100:00100000:0.0:1788845048.416451:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641850368:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.416472:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.416492:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7e4 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.416531:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845048.416543:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:0.0:1788845048.416551:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.416565:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845048.416568:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845048.416572:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006000, rc 0) 00000100:00100000:0.0:1788845048.416647:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641850368:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 197us (610us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.416654:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1661 00000100:00100000:2.0:1788845048.420374:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.420678:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641850496 00000100:00100000:0.0:1788845048.420688:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641850496 00000100:00100000:0.0:1788845048.420790:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1257 00000100:00100000:0.0:1788845048.420801:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641850496:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.420819:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.420837:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287af3b1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7eb expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.420934:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a07b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660424832:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845048.421008:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845048.425174:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a07b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660424832:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845048.425221:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287af3b1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7eb expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.425248:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007de6eff5/0xb2187301287af3b1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7eb expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.425262:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845048.425323:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641850496:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4523us (4957us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.425332:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1257 00000100:00100000:1.0:1788845048.428492:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.428774:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641850624 00000100:00100000:2.0:1788845048.428783:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641850624 00000100:00100000:2.0:1788845048.428911:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1258 00000100:00100000:2.0:1788845048.428922:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641850624:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.428938:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.428955:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287af3b8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7f2 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.429009:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a034f21f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660424960:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.429102:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.431569:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a034f21f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660424960:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845048.431619:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287af3b8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f2 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.431644:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7e5fdab/0xb2187301287af3b8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f2 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.431657:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864200, rc 0) 00000100:00100000:1.0:1788845048.431762:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641850624:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2841us (3277us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.431770:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1258 00000100:00100000:3.0:1788845048.434792:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.434875:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641850880 00000100:00100000:1.0:1788845048.434965:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641850880 00000100:00100000:1.0:1788845048.435063:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1662 00000100:00100000:1.0:1788845048.435074:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03446300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641850880:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845048.435154:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845048.435182:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af253 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa68d expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.435208:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845048.435212:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845048.435493:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.435508:0:11036:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.435516:0:11036:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.435549:0:11036:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 10 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.435567:0:11036:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 10 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.435593:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660425088:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845048.435677:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.440108:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.440210:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641851008 00000100:00100000:0.0:1788845048.440219:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641851008 00010000:00010000:0.0:1788845048.440268:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 11 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.440311:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 566 00000100:00100000:0.0:1788845048.440321:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641851008:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845048.440339:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845048.440346:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 11 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.440374:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 11 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.440397:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845048.440402:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.440418:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.440426:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.440446:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845048.440515:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641851008:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 195us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.440522:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 566 00010000:00010000:1.0:1788845048.441498:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af230 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa671 expref: 10 pid: 11036 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.441517:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660425088:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845048.441536:0:11036:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.441546:0:11036:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.441552:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845048.441671:0:11036:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967881 00000004:00010000:1.0:1788845048.441719:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 528384 -> 528384 00000004:00010000:1.0:1788845048.441723:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845048 -> 1788845048 00000004:00010000:1.0:1788845048.441725:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845048 -> 1788845047 00000004:00010000:1.0:1788845048.441727:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845048 -> 1788845048 00000004:00010000:1.0:1788845048.441729:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:1.0:1788845048.441735:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000b8606d8 reply state 000000005e9e7164 transno 4294967882 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.441743:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845048.441830:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03446300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641850880:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 6758us (7046us total) trans 4294967882 rc 0/0 00000100:00100000:1.0:1788845048.441838:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1662 00000020:00080000:1.0:1788845048.441843:0:11036:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788845048.444667:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.444685:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.444693:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.444700:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845048.444705:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.444713:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.444730:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.444733:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845048.444737:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.444743:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af3bf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.447101:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.447551:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641851136 00000100:00100000:0.0:1788845048.447561:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641851136 00000100:00100000:0.0:1788845048.447629:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1663 00000100:00100000:0.0:1788845048.447640:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641851136:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.447663:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.447667:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788845048.447706:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e400/0xb2187301287af395 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7cf expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.447723:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845048.447727:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845048.447740:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa7f9 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.447763:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.447777:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.447808:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 9 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.447824:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 9 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.447837:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a07b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660425216:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845048.447947:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18180 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.452219:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.452347:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641851264 00000100:00100000:1.0:1788845048.452356:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641851264 00010000:00010000:1.0:1788845048.452394:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 10 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.452441:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 567 00000100:00100000:1.0:1788845048.452452:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641851264:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845048.452468:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845048.452494:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 10 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.452517:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 10 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.452541:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845048.452548:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.452559:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.452568:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.452617:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.452631:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660425344:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845048.452710:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00010000:00010000:0.0:1788845048.454512:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af3aa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7e4 expref: 10 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.454534:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a07b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660425216:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845048.454561:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.454586:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845048.454590:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845048.454593:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006a00, rc 0) 00000100:00100000:0.0:1788845048.454654:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641851136:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7015us (7582us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.454663:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1663 00000100:00100000:1.0:1788845048.457105:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660425344:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845048.457131:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845048.457182:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2369:x1875739641851264:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 4732us (4973us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845048.457193:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 567 00000100:00100000:2.0:1788845048.462847:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.462997:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641851392 00000100:00100000:3.0:1788845048.463008:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641851392 00000100:00100000:3.0:1788845048.463074:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1259 00000100:00100000:3.0:1788845048.463096:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:14891:x1875739641851392:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.463120:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.463141:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af3cd lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa800 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.463172:0:6685:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.463182:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4:0x0].0x0 since=1s v=(3 13) 00010000:00010000:3.0:1788845048.463187:0:6685:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 13) 00010000:00010000:3.0:1788845048.463192:0:6685:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af3cd lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa800 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.463225:0:6685:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.463243:0:6685:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 6 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.463257:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009684380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660425472:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845048.463336:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e220c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.470294:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.470477:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641851520 00000100:00100000:0.0:1788845048.470487:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641851520 00002000:00100000:0.0:1788845048.470541:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1233348) req@ffff94a009684700 x1875739641851520/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.470553:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009684700 with x1875739641851520 ext(1216512->1233348) 00010000:00010000:0.0:1788845048.470562:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 7 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.470610:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 7 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845048.470619:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a009684700 00000100:00100000:0.0:1788845048.470665:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 38 00000100:00100000:0.0:1788845048.470676:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+7:2370:x1875739641851520:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:1.0:1788845048.471328:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009684380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660425472:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845048.471361:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af3cd lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa800 expref: 6 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.471380:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fe00, rc 0) 00000100:00100000:1.0:1788845048.471432:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034f21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:14891:x1875739641851392:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 8337us (8594us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845048.471439:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1259 00000100:00100000:0.0:1788845048.475776:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2370:x1875739641851520:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 5092us (5481us total) trans 4294967517 rc 0/0 00000100:00100000:0.0:1788845048.475790:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 38 00002000:00100000:0.0:1788845048.475802:0:6690:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (1216512->1233348) req@ffff94a009684700 x1875739641851520/t4294967517(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/448 e 0 to 0 dl 1788845059 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.475813:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a009684700 with x1875739641851520 ext(1216512->1233348) 00010000:00010000:0.0:1788845048.475823:0:6690:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 9 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.475848:0:6690:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 9 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788845048.475861:0:6690:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff94a009684700 00000100:00100000:3.0:1788845048.479789:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.479923:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641851648 00000100:00100000:0.0:1788845048.479933:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641851648 00010000:00010000:0.0:1788845048.479970:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 9 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.480012:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 568 00000100:00100000:0.0:1788845048.480021:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2367:x1875739641851648:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845048.480036:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845048.480061:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 9 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.480347:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 9 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.480377:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af0e0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa54b expref: 9 pid: 6685 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.480396:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 13) 00010000:00010000:0.0:1788845048.480401:0:13304:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(4 13) 00010000:00010000:0.0:1788845048.480410:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af3cd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa800 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.480420:0:13304:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af3cd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa800 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.480527:0:13304:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000e6ab06b/0xb2187301287af3cd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa800 expref: 5 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.480543:0:13304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a07b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660425600:192.168.201.29@tcp:105: 00000100:00100000:0.0:1788845048.480689:0:13304:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18180 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.484569:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.484965:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641851776 00000100:00100000:2.0:1788845048.484974:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641851776 00002000:00010000:2.0:1788845048.485014:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1790875->18446744073709551615) 00002000:00100000:2.0:1788845048.485019:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5dc380 with x1875739641851776 ext(1790875->18446744073709551615) 00002000:00010000:2.0:1788845048.485030:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5dc380. 00000100:00100000:2.0:1788845048.485063:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 637 00000100:00100000:2.0:1788845048.485072:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641851776:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:2.0:1788845048.485291:0:13321:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1233349 -> 1790875 00002000:00010000:2.0:1788845048.485295:0:13321:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845048 -> 1788845048 00002000:00010000:2.0:1788845048.485298:0:13321:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845048 -> 1788845047 00002000:00010000:2.0:1788845048.485300:0:13321:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845048 -> 1788845048 00002000:00010000:2.0:1788845048.485302:0:13321:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 137 -> 137 00000100:00100000:2.0:1788845048.485353:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641851776:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 282us (793us total) trans 4294967518 rc 0/0 00000100:00100000:2.0:1788845048.485364:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 637 00002000:00010000:2.0:1788845048.485367:0:13321:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (1790875->18446744073709551615) 00002000:00100000:2.0:1788845048.485369:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a03d5dc380 with x1875739641851776 ext(1790875->18446744073709551615) 00002000:00010000:2.0:1788845048.485374:0:13321:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a03d5dc380. 00000100:00100000:0.0:1788845048.485720:0:13304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a07b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:13304:1875739660425600:192.168.201.29@tcp:105: 00010000:00010000:0.0:1788845048.485741:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845048.485764:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641851648:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 5745us (5987us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.485772:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 568 00000100:00100000:2.0:1788845048.488722:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.489268:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641851904 00000100:00100000:3.0:1788845048.489278:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641851904 00000100:00100000:3.0:1788845048.489375:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1664 00000100:00100000:3.0:1788845048.489387:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:14891:x1875739641851904:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.489407:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.489427:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af3d4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa807 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.489521:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.489536:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.489548:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845048.489573:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1790875 00000004:00200000:3.0:1788845048.489588:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845048.489608:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.489632:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af3d4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.489650:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 10 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.489658:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.489661:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845048.489664:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e800, rc 0) 00000100:00100000:3.0:1788845048.489731:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641851904:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 345us (1017us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.489738:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1664 00000100:00100000:2.0:1788845048.492225:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.492402:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641852032 00000100:00100000:3.0:1788845048.492411:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641852032 00000100:00100000:3.0:1788845048.492467:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1665 00000100:00100000:3.0:1788845048.492476:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641852032:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.492492:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.492511:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af3e2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa80e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.492542:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.492588:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009685c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660425728:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.492657:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22480 going to sleep for 11 seconds 00000100:00100000:0.0:1788845048.494333:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000004:00010000:3.0:1788845048.494440:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845047 -> 1788845048 00000100:00100000:3.0:1788845048.494470:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009685c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660425728:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845048.494490:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:3.0:1788845048.494505:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af3e2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa80e expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.494521:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.494524:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845048.494528:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af3e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa80e expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.494542:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:3.0:1788845048.494600:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641852032:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2124us (2381us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.494607:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1665 00000100:00100000:1.0:1788845048.494918:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641852160 00000100:00100000:1.0:1788845048.494926:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641852160 00000100:00100000:1.0:1788845048.494972:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1260 00000100:00100000:1.0:1788845048.494980:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a07b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641852160:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.494993:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.495006:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af3e9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa815 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.495051:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660425856:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.495108:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fb40 going to sleep for 11 seconds 00002000:00010000:2.0:1788845048.498100:0:7983:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845047 -> 1788845048 00000100:00100000:2.0:1788845048.498131:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660425856:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845048.498168:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af3e9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa815 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.498192:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee7a3085/0xb2187301287af3e9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa815 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.498205:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:2.0:1788845048.498263:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a07b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641852160:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3282us (3935us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.498271:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1260 00000100:00100000:3.0:1788845048.501226:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.501349:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641852288 00000100:00100000:1.0:1788845048.501358:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641852288 00000100:00100000:1.0:1788845048.501410:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1666 00000100:00100000:1.0:1788845048.501418:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641852288:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.501438:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.501465:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af3f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa81c expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.501501:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.501540:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660425984:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.501623:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.504229:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.504386:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660425984:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845048.504407:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:1.0:1788845048.504425:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af3f0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa81c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.504445:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845048.504448:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845048.504452:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af3f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa81c expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.504477:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:1.0:1788845048.504532:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641852288:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3114us (3314us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.504540:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1666 00000100:00100000:0.0:1788845048.504665:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641852416 00000100:00100000:0.0:1788845048.504672:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641852416 00000100:00100000:0.0:1788845048.504714:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1261 00000100:00100000:0.0:1788845048.504723:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641852416:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.504737:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.504752:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008561da8b/0xb2187301287af3f7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa823 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.504800:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a04700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660426112:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845048.504848:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18180 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.507236:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a04700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660426112:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845048.507270:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008561da8b/0xb2187301287af3f7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa823 expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.507292:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008561da8b/0xb2187301287af3f7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa823 expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.507307:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007800, rc 0) 00000100:00100000:2.0:1788845048.507364:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641852416:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2641us (3142us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.507372:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1261 00000100:00100000:3.0:1788845048.512064:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.512194:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641852544 00000100:00100000:2.0:1788845048.512204:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641852544 00000100:00100000:2.0:1788845048.512269:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 638 00000100:00100000:2.0:1788845048.512280:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641852544:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:2.0:1788845048.512544:0:6689:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1790875 -> 1790875 00002000:00010000:2.0:1788845048.512549:0:6689:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845048 -> 1788845048 00002000:00010000:2.0:1788845048.512552:0:6689:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845048 -> 1788845047 00002000:00010000:2.0:1788845048.512554:0:6689:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845048 -> 1788845048 00002000:00010000:2.0:1788845048.512556:0:6689:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 137 -> 137 00000100:00100000:2.0:1788845048.512633:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641852544:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 355us (578us total) trans 4294967519 rc 0/0 00000100:00100000:2.0:1788845048.512640:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 638 00000100:00100000:3.0:1788845048.516174:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.516220:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.516352:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641852800 00000100:00100000:2.0:1788845048.516362:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641852800 00000100:00100000:2.0:1788845048.516413:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1262 00000100:00100000:2.0:1788845048.516422:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641852800:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.516442:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.516470:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af3fe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa831 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.516538:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660426240:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.516630:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845048.517329:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641852672 00000100:00100000:0.0:1788845048.517335:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641852672 00000100:00100000:0.0:1788845048.517378:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1667 00000100:00100000:0.0:1788845048.517387:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641852672:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.517400:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.517416:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af405 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa82a expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.517445:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.517484:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a06a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660426368:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845048.517535:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00002000:00010000:2.0:1788845048.520652:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845047 -> 1788845048 00000100:00100000:2.0:1788845048.520680:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660426240:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845048.520718:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af3fe lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa831 expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.520741:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a9c54d9d/0xb2187301287af3fe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa831 expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.520755:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865c00, rc 0) 00000100:00100000:2.0:1788845048.520817:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641852800:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4396us (4597us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.520826:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1262 00000100:00100000:0.0:1788845048.521355:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a06a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660426368:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845048.521371:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:0.0:1788845048.521383:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af405 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa82a expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.521397:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845048.521400:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845048.521404:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af405 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa82a expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.521416:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006200, rc 0) 00000100:00100000:0.0:1788845048.521459:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641852672:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4073us (5295us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.521466:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1667 00000100:00100000:3.0:1788845048.526293:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.526343:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.526489:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641852928 00000100:00100000:2.0:1788845048.526498:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641852928 00000100:00100000:2.0:1788845048.526560:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1668 00000100:00100000:2.0:1788845048.526571:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641852928:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.526601:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.526620:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af40c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa838 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.526655:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.526695:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660426496:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.526781:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.527004:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641853056 00000100:00100000:1.0:1788845048.527008:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641853056 00000100:00100000:1.0:1788845048.527036:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1263 00000100:00100000:1.0:1788845048.527042:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641853056:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.527051:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.527063:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287af413 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa83f expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.527098:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660426624:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.527146:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.531646:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660426624:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845048.531687:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287af413 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa83f expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.531712:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287af413 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa83f expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.531724:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:1.0:1788845048.531786:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641853056:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4743us (5441us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.531795:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1263 00000100:00100000:2.0:1788845048.534235:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660426496:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845048.534275:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:2.0:1788845048.534294:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af40c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa838 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.534315:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.534318:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.534323:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af40c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa838 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.534338:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865c00, rc 0) 00000100:00100000:2.0:1788845048.534406:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641852928:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7836us (8122us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.534415:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1668 00000100:00100000:0.0:1788845048.550347:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.550471:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.551055:0:13321:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641853184 00000100:00100000:2.0:1788845048.551065:0:13321:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641853184 00002000:00100000:2.0:1788845048.551133:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1794047) req@ffff949f03a06a00 x1875739641853184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.551143:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a06a00 with x1875739641853184 ext(1048576->1794047) 00002000:00010000:2.0:1788845048.551149:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a06a00 00000100:00100000:2.0:1788845048.551185:0:13321:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 639 00000100:00100000:2.0:1788845048.551196:0:13321:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a06a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641853184:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845048.552320:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641853312 00000100:00100000:1.0:1788845048.552330:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641853312 00002000:00100000:1.0:1788845048.552381:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a04700 x1875739641853312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845048.552392:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a04700 with x1875739641853312 00000100:00100000:1.0:1788845048.552416:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1114 00000100:00100000:1.0:1788845048.552424:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a04700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641853312:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845048.580298:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a04700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641853312:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 27865us (29817us total) trans 0 rc 528384/528384 00000100:00100000:1.0:1788845048.580312:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1114 00002000:00100000:1.0:1788845048.580324:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a04700 x1875739641853312/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 528384/528384 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845048.580339:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a04700 with x1875739641853312 00000100:00100000:2.0:1788845048.585460:0:13321:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a06a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641853184:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 34256us (35111us total) trans 0 rc 745472/745472 00000100:00100000:2.0:1788845048.585476:0:13321:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 639 00002000:00100000:2.0:1788845048.585488:0:13321:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4:0x0] (1048576->1794047) req@ffff949f03a06a00 x1875739641853184/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 745472/745472 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.585499:0:13321:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff949f03a06a00 with x1875739641853184 ext(1048576->1794047) 00002000:00010000:2.0:1788845048.585505:0:13321:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff949f03a06a00 00000100:00100000:3.0:1788845048.589195:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.589250:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.589734:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641853568 00000100:00100000:1.0:1788845048.589744:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641853568 00000100:00100000:1.0:1788845048.589808:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1264 00000100:00100000:1.0:1788845048.589819:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641853568:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.589838:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.589859:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287af41a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa84d expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.589961:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660426752:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.590036:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.590619:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641853440 00000100:00100000:2.0:1788845048.590626:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641853440 00000100:00100000:2.0:1788845048.590672:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1669 00000100:00100000:2.0:1788845048.590680:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641853440:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.590693:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.590708:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af421 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa846 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.590737:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.590763:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660426880:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.590812:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.593670:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660426752:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845048.593710:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287af41a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa84d expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.593735:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e21a3168/0xb2187301287af41a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa84d expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.593748:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1600, rc 0) 00000100:00100000:1.0:1788845048.593808:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2369:x1875739641853568:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3990us (4557us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.593817:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1264 00000100:00100000:2.0:1788845048.596968:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660426880:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845048.596994:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:2.0:1788845048.597013:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af421 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa846 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.597039:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.597042:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.597047:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af421 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa846 expref: 11 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.597059:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865c00, rc 0) 00000100:00100000:2.0:1788845048.597262:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641853440:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6572us (8066us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.597272:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1669 00000100:00100000:0.0:1788845048.600757:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.600806:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.600917:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641853824 00000100:00100000:2.0:1788845048.600927:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641853824 00000100:00100000:2.0:1788845048.600988:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1265 00000100:00100000:2.0:1788845048.600999:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641853824:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.601017:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.601035:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af428 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa85b expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.601106:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660427008:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.601190:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.603191:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641853696 00000100:00100000:3.0:1788845048.603205:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641853696 00000100:00100000:0.0:1788845048.603349:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660427008:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.603369:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1670 00000100:00100000:3.0:1788845048.603382:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641853696:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.603388:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af428 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa85b expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.603402:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.603412:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df9c0caa/0xb2187301287af428 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa85b expref: 9 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.603418:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af42f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa854 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.603425:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00010000:00010000:3.0:1788845048.603454:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.603479:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660427136:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845048.603624:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2368:x1875739641853824:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2626us (2814us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.603632:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1265 00000100:00100000:3.0:1788845048.603712:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22480 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.605859:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660427136:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845048.606001:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 528384 00010000:00010000:3.0:1788845048.606019:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af42f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa854 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.606039:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.606042:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845048.606047:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af42f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa854 expref: 11 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.606062:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:2.0:1788845048.606209:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641853696:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2828us (5459us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.606217:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1670 00000100:00100000:1.0:1788845048.608809:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.608880:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641853952 00000100:00100000:0.0:1788845048.608906:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641853952 00000100:00100000:0.0:1788845048.608962:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1671 00000100:00100000:0.0:1788845048.608971:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641853952:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.608990:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.609008:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa862 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.609034:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.609044:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.609080:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.609096:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 15 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.609110:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399e300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660427264:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845048.609186:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.613368:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.613522:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641854080 00000100:00100000:1.0:1788845048.613532:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641854080 00010000:00010000:1.0:1788845048.613583:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.613628:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 569 00000100:00100000:1.0:1788845048.613639:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a14380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641854080:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845048.613654:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845048.613679:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.613699:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.613720:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845048.613727:0:5825:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.613737:0:5825:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.613746:0:5825:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.613784:0:5825:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.613797:0:5825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660427392:192.168.201.29@tcp:105: 00000100:00100000:1.0:1788845048.613904:0:5825:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00010000:00010000:0.0:1788845048.614174:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af3c6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa7f9 expref: 16 pid: 5840 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.614190:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399e300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660427264:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845048.614212:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 11 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.614224:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845048.614226:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845048.614229:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006800, rc 0) 00000100:00100000:0.0:1788845048.614262:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641853952:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 5291us (5460us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.614269:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1671 00000100:00100000:1.0:1788845048.616468:0:5825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5825:1875739660427392:192.168.201.29@tcp:105: 00010000:00010000:1.0:1788845048.616494:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845048.616544:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a14380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641854080:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 2907us (3198us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845048.616555:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 569 00000100:00100000:1.0:1788845048.619685:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.619811:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641854208 00000100:00100000:3.0:1788845048.619821:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641854208 00000100:00100000:3.0:1788845048.620123:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1266 00000100:00100000:2.0:1788845048.620195:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641854208:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.620218:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.620236:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af43d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa869 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.620296:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660427520:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.620388:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845048.629929:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660427520:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845048.629968:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af43d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa869 expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.629998:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af43d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa869 expref: 9 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.630009:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:0.0:1788845048.630061:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2370:x1875739641854208:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 9927us (10385us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.630070:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1266 00000100:00100000:3.0:1788845048.632744:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.633307:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641854336 00000100:00100000:1.0:1788845048.633316:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641854336 00000100:00100000:1.0:1788845048.633490:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1267 00000100:00100000:1.0:1788845048.633499:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03450bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641854336:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.633516:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.633531:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287af444 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa870 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.633593:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660427648:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.633669:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.635148:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660427648:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845048.635178:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287af444 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa870 expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.635196:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000022efae93/0xb2187301287af444 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa870 expref: 9 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.635206:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:3.0:1788845048.635260:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03450bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641854336:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 1754us (2516us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.635268:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1267 00000100:00100000:2.0:1788845048.644896:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.645409:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641854464 00000100:00100000:1.0:1788845048.645418:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641854464 00000100:00100000:1.0:1788845048.645605:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1268 00000100:00100000:1.0:1788845048.645618:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641854464:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.645637:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.645654:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287af44b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa877 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.645704:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0346f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660427776:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.645777:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a02dd8fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.647809:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0346f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660427776:192.168.201.29@tcp:106: 00010000:00010000:1.0:1788845048.647841:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287af44b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa877 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.647861:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0892274/0xb2187301287af44b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa877 expref: 9 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.647871:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0e00, rc 0) 00000100:00100000:1.0:1788845048.647957:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+8:2367:x1875739641854464:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2341us (3083us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.647965:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1268 00000100:00100000:0.0:1788845048.652901:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.652980:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641854592 00000100:00100000:2.0:1788845048.652990:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641854592 00002000:00100000:2.0:1788845048.653031:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399e300 x1875739641854592/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.653040:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399e300 with x1875739641854592 00000100:00100000:2.0:1788845048.653066:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1115 00000100:00100000:2.0:1788845048.653073:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641854592:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.653716:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641854592:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 641us (837us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.653723:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1115 00002000:00100000:2.0:1788845048.653729:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399e300 x1875739641854592/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.653735:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399e300 with x1875739641854592 00000100:00100000:1.0:1788845048.663562:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.663666:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641854720 00000100:00100000:3.0:1788845048.663676:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641854720 00002000:00100000:3.0:1788845048.663723:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8000 x1875739641854720/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.663733:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8000 with x1875739641854720 00000100:00100000:3.0:1788845048.663806:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1116 00000100:00100000:3.0:1788845048.663817:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641854720:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.664299:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641854720:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 482us (744us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.664307:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1116 00002000:00100000:3.0:1788845048.664316:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8000 x1875739641854720/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.664333:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8000 with x1875739641854720 00000100:00100000:2.0:1788845048.668762:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.668865:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641854848 00000100:00100000:0.0:1788845048.668876:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641854848 00002000:00100000:0.0:1788845048.669002:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a16300 x1875739641854848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.669013:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a16300 with x1875739641854848 00000100:00100000:0.0:1788845048.669205:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1117 00000100:00100000:0.0:1788845048.669217:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641854848:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.669495:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641854848:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 279us (741us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.669502:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1117 00002000:00100000:0.0:1788845048.669509:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a16300 x1875739641854848/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.669517:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a16300 with x1875739641854848 00000100:00100000:1.0:1788845048.672762:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.673262:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641854976 00000100:00100000:2.0:1788845048.673279:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641854976 00002000:00100000:2.0:1788845048.673332:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9880 x1875739641854976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.673349:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9880 with x1875739641854976 00000100:00100000:2.0:1788845048.673377:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1118 00000100:00100000:2.0:1788845048.673386:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641854976:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.673629:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641854976:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 243us (876us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.673636:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1118 00002000:00100000:2.0:1788845048.673642:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9880 x1875739641854976/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.673650:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9880 with x1875739641854976 00000100:00100000:1.0:1788845048.676807:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.676910:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641855104 00000100:00100000:2.0:1788845048.676921:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641855104 00002000:00100000:2.0:1788845048.676971:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa680 x1875739641855104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.676982:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa680 with x1875739641855104 00000100:00100000:2.0:1788845048.677006:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1119 00000100:00100000:2.0:1788845048.677016:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641855104:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.677276:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641855104:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 260us (477us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.677285:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1119 00002000:00100000:2.0:1788845048.677292:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa680 x1875739641855104/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.677299:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa680 with x1875739641855104 00000100:00100000:2.0:1788845048.682275:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.682375:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641855232 00000100:00100000:0.0:1788845048.682385:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641855232 00002000:00100000:0.0:1788845048.682431:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15500 x1875739641855232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.682443:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15500 with x1875739641855232 00000100:00100000:0.0:1788845048.682568:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1120 00000100:00100000:0.0:1788845048.682588:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641855232:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.682868:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641855232:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 282us (602us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.682876:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1120 00002000:00100000:0.0:1788845048.682909:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03a15500 x1875739641855232/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.682916:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03a15500 with x1875739641855232 00000100:00100000:1.0:1788845048.686093:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.686603:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641855360 00000100:00100000:0.0:1788845048.686614:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641855360 00002000:00100000:0.0:1788845048.686664:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9c00 x1875739641855360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.686674:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9c00 with x1875739641855360 00000100:00100000:0.0:1788845048.686713:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1121 00000100:00100000:0.0:1788845048.686723:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641855360:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.686989:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641855360:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 267us (905us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.686996:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1121 00002000:00100000:0.0:1788845048.687004:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9c00 x1875739641855360/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.687011:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9c00 with x1875739641855360 00000100:00100000:1.0:1788845048.690857:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.690998:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641855488 00000100:00100000:3.0:1788845048.691009:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641855488 00002000:00100000:3.0:1788845048.691071:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9f80 x1875739641855488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.691083:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9f80 with x1875739641855488 00000100:00100000:3.0:1788845048.691123:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1122 00000100:00100000:3.0:1788845048.691134:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641855488:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.691698:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641855488:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 564us (852us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.691706:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1122 00002000:00100000:3.0:1788845048.691714:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9f80 x1875739641855488/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.691722:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9f80 with x1875739641855488 00000100:00100000:1.0:1788845048.700512:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.700726:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641855616 00000100:00100000:3.0:1788845048.700738:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641855616 00002000:00100000:3.0:1788845048.700790:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9180 x1875739641855616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.700800:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9180 with x1875739641855616 00000100:00100000:3.0:1788845048.700843:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1123 00000100:00100000:3.0:1788845048.700853:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641855616:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.701144:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641855616:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 292us (640us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.701151:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1123 00002000:00100000:3.0:1788845048.701158:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f9180 x1875739641855616/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.701167:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f9180 with x1875739641855616 00000100:00100000:1.0:1788845048.705004:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.705120:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641855744 00000100:00100000:2.0:1788845048.705131:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641855744 00002000:00100000:2.0:1788845048.705180:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fbb80 x1875739641855744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.705191:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fbb80 with x1875739641855744 00000100:00100000:2.0:1788845048.705233:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1124 00000100:00100000:2.0:1788845048.705243:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641855744:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.705671:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641855744:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 428us (675us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.705679:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1124 00002000:00100000:2.0:1788845048.705686:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fbb80 x1875739641855744/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.705693:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fbb80 with x1875739641855744 00000100:00100000:1.0:1788845048.709563:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.709656:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641855872 00000100:00100000:3.0:1788845048.709666:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641855872 00002000:00100000:3.0:1788845048.709711:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa300 x1875739641855872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.709721:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa300 with x1875739641855872 00000100:00100000:3.0:1788845048.709759:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1125 00000100:00100000:3.0:1788845048.709768:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641855872:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.710056:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641855872:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 288us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.710062:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1125 00002000:00100000:3.0:1788845048.710069:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346fa300 x1875739641855872/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.710076:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346fa300 with x1875739641855872 00000100:00100000:1.0:1788845048.714964:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.715135:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641856000 00000100:00100000:0.0:1788845048.715147:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641856000 00002000:00100000:0.0:1788845048.715195:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8380 x1875739641856000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.715206:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8380 with x1875739641856000 00000100:00100000:0.0:1788845048.715247:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1126 00000100:00100000:0.0:1788845048.715608:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0346f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641856000:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.715880:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0346f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641856000:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 623us (925us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.715924:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1126 00002000:00100000:0.0:1788845048.715932:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0346f8380 x1875739641856000/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.715940:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0346f8380 with x1875739641856000 00000100:00100000:1.0:1788845048.720698:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.720798:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641856128 00000100:00100000:3.0:1788845048.720808:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641856128 00002000:00100000:3.0:1788845048.720860:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93b80 x1875739641856128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.720872:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93b80 with x1875739641856128 00000100:00100000:3.0:1788845048.720959:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1127 00000100:00100000:3.0:1788845048.720970:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641856128:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.721248:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2369:x1875739641856128:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 280us (560us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845048.721698:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1127 00002000:00100000:3.0:1788845048.721708:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93b80 x1875739641856128/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.721716:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93b80 with x1875739641856128 00000100:00100000:1.0:1788845048.731293:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.731554:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641856256 00000100:00100000:0.0:1788845048.731565:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641856256 00002000:00100000:0.0:1788845048.731626:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93100 x1875739641856256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.731637:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93100 with x1875739641856256 00000100:00100000:0.0:1788845048.731681:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1128 00000100:00100000:0.0:1788845048.731690:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641856256:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:0.0:1788845048.731957:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2367:x1875739641856256:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 267us (679us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845048.731965:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1128 00002000:00100000:0.0:1788845048.731972:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d93100 x1875739641856256/t0(0) o3->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/440 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845048.731983:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d93100 with x1875739641856256 00000100:00100000:1.0:1788845048.736509:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.736650:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641856384 00000100:00100000:2.0:1788845048.736661:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641856384 00000100:00100000:2.0:1788845048.736714:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1672 00000100:00100000:2.0:1788845048.736722:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641856384:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.736742:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.736760:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af452 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa87e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.736833:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af459 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.736848:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af459 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.736863:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af459 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845048.736972:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1790875 00000004:00200000:2.0:1788845048.736977:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845048.736994:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af459 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.737018:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af452 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa87e expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.737033:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af459 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa87e expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.737041:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.737044:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845048.737047:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845048.737106:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641856384:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 385us (606us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.737112:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1672 00000100:00100000:1.0:1788845048.741988:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.742304:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641856512 00000100:00100000:2.0:1788845048.742531:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641856512 00000100:00100000:2.0:1788845048.742603:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1673 00000100:00100000:2.0:1788845048.742614:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641856512:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.742634:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.742654:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af460 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa885 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.742693:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.742734:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660427904:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.742825:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000004:00010000:2.0:1788845048.747024:0:5841:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 528384 -> 887101 00000100:00100000:2.0:1788845048.747060:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660427904:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845048.747085:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:2.0:1788845048.747113:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af460 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa885 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.747132:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.747135:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.747140:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af460 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa885 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.747154:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865200, rc 0) 00000100:00100000:2.0:1788845048.747217:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641856512:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4605us (5239us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.747226:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1673 00000100:00100000:1.0:1788845048.751118:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.751960:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641856640 00000100:00100000:2.0:1788845048.751972:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641856640 00000100:00100000:2.0:1788845048.752068:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1674 00000100:00100000:2.0:1788845048.752079:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641856640:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.752110:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.752131:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af467 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa88c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.752171:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.752210:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660428032:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.752650:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.755669:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660428032:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845048.755697:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:3.0:1788845048.755716:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af467 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa88c expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.755737:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.755739:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845048.755745:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af467 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa88c expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.755759:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865e00, rc 0) 00000100:00100000:3.0:1788845048.755820:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641856640:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3742us (4710us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.755828:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1674 00000100:00100000:1.0:1788845048.759825:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.759925:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641856768 00000100:00100000:0.0:1788845048.759935:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641856768 00000100:00100000:0.0:1788845048.759994:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1675 00000100:00100000:0.0:1788845048.760004:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641856768:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.760028:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.760048:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af46e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa893 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.760086:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.760139:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660428160:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845048.760223:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788845048.763161:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660428160:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845048.763189:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:0.0:1788845048.763208:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af46e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa893 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.763230:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845048.763233:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845048.763238:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af46e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa893 expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845048.763253:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845048.763491:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:2369:x1875739641856768:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3487us (3676us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.763501:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1675 00000100:00100000:1.0:1788845048.774315:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.774494:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641856896 00000100:00100000:2.0:1788845048.774503:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641856896 00002000:00100000:2.0:1788845048.774555:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1318911) req@ffff94a019d91500 x1875739641856896/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.774565:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d91500 with x1875739641856896 ext(1253376->1318911) 00002000:00010000:2.0:1788845048.774598:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d91500 00000100:00100000:2.0:1788845048.774632:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 640 00000100:00100000:2.0:1788845048.774641:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641856896:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:2.0:1788845048.787082:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641856896:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 12430us (12764us total) trans 4294967520 rc 0/0 00000100:00100000:2.0:1788845048.787106:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 640 00002000:00100000:2.0:1788845048.787121:0:6689:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (1253376->1318911) req@ffff94a019d91500 x1875739641856896/t4294967520(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/448 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.787131:0:6689:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a019d91500 with x1875739641856896 ext(1253376->1318911) 00002000:00010000:2.0:1788845048.787137:0:6689:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a019d91500 00000100:00100000:2.0:1788845048.790920:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.791017:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641857024 00000100:00100000:1.0:1788845048.791027:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641857024 00000100:00100000:1.0:1788845048.791087:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1269 00000100:00100000:1.0:1788845048.791110:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+10:2370:x1875739641857024:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:0.0:1788845048.843463:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967517 is committed 00000001:00080000:0.0:1788845048.843488:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967520 is committed 00000100:00100000:1.0:1788845048.843666:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641857024:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 52553us (52750us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845048.843678:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1269 00000100:00100000:2.0:1788845048.848440:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.848544:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641857152 00000100:00100000:0.0:1788845048.848554:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641857152 00000100:00100000:0.0:1788845048.848616:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1270 00000100:00100000:0.0:1788845048.848626:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641857152:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845048.848646:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845048.848663:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008561da8b/0xb2187301287af475 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa89a expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845048.848731:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0399e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660428288:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845048.848819:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c189c0 going to sleep for 11 seconds 00002000:00010000:2.0:1788845048.855147:0:6682:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks: 137 -> 179 00000100:00100000:2.0:1788845048.855171:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0399e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660428288:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845048.855200:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008561da8b/0xb2187301287af475 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa89a expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.855218:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008561da8b/0xb2187301287af475 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa89a expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.855228:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007800, rc 0) 00000100:00100000:2.0:1788845048.855283:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641857152:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6658us (6853us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.855292:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1270 00000100:00100000:2.0:1788845048.858455:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.858566:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641857280 00000100:00100000:3.0:1788845048.858587:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641857280 00000100:00100000:3.0:1788845048.859105:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1271 00000100:00100000:3.0:1788845048.859116:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641857280:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.859134:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.859153:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287af47c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8a1 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.859201:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660428416:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.859273:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.866078:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660428416:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845048.866132:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287af47c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8a1 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.866158:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287af47c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8a1 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.866174:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845048.866361:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641857280:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7244us (7913us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845048.866369:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1271 00000100:00100000:2.0:1788845048.872848:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.873098:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641857408 00000100:00100000:1.0:1788845048.873108:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641857408 00000100:00100000:1.0:1788845048.873167:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1676 00000100:00100000:1.0:1788845048.873177:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641857408:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845048.873200:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845048.873220:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af483 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8a8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.873329:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845048.873376:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a019d90000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660428544:192.168.201.29@tcp:106: 00000100:00100000:1.0:1788845048.873662:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4900 going to sleep for 11 seconds 00000100:00100000:1.0:1788845048.880897:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a019d90000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660428544:192.168.201.29@tcp:106: 00002000:00010000:1.0:1788845048.880929:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:1.0:1788845048.880961:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af483 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8a8 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.880988:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845048.880991:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845048.880996:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af483 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8a8 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845048.881011:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0800, rc 0) 00000100:00100000:1.0:1788845048.881158:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:2367:x1875739641857408:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 7982us (8318us total) trans 0 rc 301/301 00000100:00100000:1.0:1788845048.881167:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1676 00000100:00100000:0.0:1788845048.885189:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.885280:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641857536 00000100:00100000:3.0:1788845048.885291:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641857536 00000100:00100000:3.0:1788845048.885351:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1677 00000100:00100000:3.0:1788845048.885361:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641857536:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845048.885382:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845048.885403:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af48a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8af expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.885539:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.885594:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660428672:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845048.885760:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.888975:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660428672:192.168.201.29@tcp:106: 00002000:00010000:3.0:1788845048.889008:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:3.0:1788845048.889027:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af48a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8af expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.889047:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.889051:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845048.889056:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af48a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8af expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.889072:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:0.0:1788845048.889223:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:2368:x1875739641857536:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3862us (4342us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845048.889232:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1677 00000100:00100000:2.0:1788845048.891826:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.892146:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641857664 00000100:00100000:3.0:1788845048.892155:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641857664 00000100:00100000:2.0:1788845048.892612:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1678 00000100:00100000:2.0:1788845048.892623:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641857664:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.892641:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.892660:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af491 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8b6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.892695:0:5839:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.892734:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660428800:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845048.892818:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.895334:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660428800:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845048.895362:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:2.0:1788845048.895382:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af491 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8b6 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.895403:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.895406:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.895411:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af491 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8b6 expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.895426:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864200, rc 0) 00000100:00100000:2.0:1788845048.895478:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:2370:x1875739641857664:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2856us (3658us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845048.895486:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1678 00000100:00100000:0.0:1788845048.901841:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.902039:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641857792 00000100:00100000:2.0:1788845048.902049:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641857792 00000100:00100000:2.0:1788845048.902102:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1679 00000100:00100000:2.0:1788845048.902112:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641857792:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845048.902132:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845048.902154:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8bd expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.902186:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.902196:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8bd expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.902230:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.902248:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 10 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845048.902275:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660428928:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845048.902353:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845048.908582:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660428928:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845048.908623:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8bd expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.908644:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845048.908646:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845048.908650:0:5840:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8bd expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.908661:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864e00, rc 0) 00000100:00100000:2.0:1788845048.908719:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641857792:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6608us (6889us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.908728:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1679 00000100:00100000:0.0:1788845048.912982:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.913132:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641857920 00000100:00100000:2.0:1788845048.913142:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641857920 00002000:00100000:2.0:1788845048.913190:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399ce00 x1875739641857920/t0(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.913208:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399ce00 with x1875739641857920 00010000:00010000:2.0:1788845048.913217:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 11 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845048.913237:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 11 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788845048.913245:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0399ce00 00000100:00100000:2.0:1788845048.913276:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 40 00000100:00100000:2.0:1788845048.913286:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2370:x1875739641857920:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845048.922874:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 887101 -> 887101 00000004:00010000:3.0:1788845048.923165:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845048 -> 1788845048 00000004:00010000:3.0:1788845048.923169:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845048 -> 1788845047 00000004:00010000:3.0:1788845048.923171:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845048 -> 1788845048 00000004:00010000:3.0:1788845048.923173:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845048.923242:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641857920:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 9953us (10267us total) trans 4294967883 rc 0/0 00000100:00100000:3.0:1788845048.923255:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 40 00002000:00100000:3.0:1788845048.923267:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399ce00 x1875739641857920/t4294967883(0) o4->5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a@192.168.201.29@tcp:154/0 lens 488/448 e 0 to 0 dl 1788845059 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.923277:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399ce00 with x1875739641857920 00010000:00010000:3.0:1788845048.923286:0:13305:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 13 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.923308:0:13305:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 13 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845048.923316:0:13305:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff949f0399ce00 00000100:00100000:0.0:1788845048.925828:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.925959:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641858048 00000100:00100000:3.0:1788845048.925971:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641858048 00010000:00010000:3.0:1788845048.926010:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 13 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.926054:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 570 00000100:00100000:3.0:1788845048.926065:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2370:x1875739641858048:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845048.926078:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845048.926103:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 13 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.926121:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 13 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.926158:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af436 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa862 expref: 13 pid: 5840 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.926174:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845048.926179:0:5826:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8bd expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.926190:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8bd expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.926199:0:5826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8bd expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845048.926239:0:5826:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8bd expref: 16 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845048.926629:0:5826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660429056:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845048.926721:0:5826:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22300 going to sleep for 11 seconds 00000100:00100000:3.0:1788845048.935004:0:5826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5826:1875739660429056:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845048.935033:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845048.935232:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641858048:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 9167us (9413us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845048.935244:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 570 00000100:00100000:2.0:1788845048.941699:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845048.944925:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641858176 00000100:00100000:0.0:1788845048.945156:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641858176 00002000:00100000:1.0:1788845048.947344:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea9180 x1875739641858176/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845048.947359:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea9180 with x1875739641858176 00000100:00100000:1.0:1788845048.947467:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1129 00000100:00100000:1.0:1788845048.947477:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641858176:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845048.947755:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641858176:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 278us (6065us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845048.947773:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1129 00002000:00100000:1.0:1788845048.947780:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea9180 x1875739641858176/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845048.947788:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea9180 with x1875739641858176 00000100:00100000:0.0:1788845048.956040:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845048.958405:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641858304 00000100:00100000:2.0:1788845048.958419:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641858304 00002000:00100000:2.0:1788845048.958478:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399df80 x1875739641858304/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.958491:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399df80 with x1875739641858304 00000100:00100000:2.0:1788845048.958581:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1130 00000100:00100000:2.0:1788845048.958592:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641858304:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845048.958868:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641858304:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (2838us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845048.958876:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1130 00002000:00100000:2.0:1788845048.958901:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0399df80 x1875739641858304/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845048.958910:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0399df80 with x1875739641858304 00000100:00100000:0.0:1788845048.973314:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.973468:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641858432 00000100:00100000:3.0:1788845048.973478:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641858432 00002000:00100000:3.0:1788845048.973537:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6680 x1875739641858432/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.973549:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6680 with x1875739641858432 00000100:00100000:3.0:1788845048.973600:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1131 00000100:00100000:3.0:1788845048.973611:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641858432:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.973876:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641858432:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (573us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845048.973906:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1131 00002000:00100000:3.0:1788845048.973913:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6680 x1875739641858432/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.973920:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6680 with x1875739641858432 00000100:00100000:2.0:1788845048.991042:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845048.991311:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641858560 00000100:00100000:1.0:1788845048.991321:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641858560 00002000:00100000:1.0:1788845048.991372:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beab100 x1875739641858560/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845048.991388:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beab100 with x1875739641858560 00000100:00100000:1.0:1788845048.991429:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1132 00000100:00100000:1.0:1788845048.991440:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03beab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641858560:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845048.991707:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03beab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641858560:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 268us (675us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845048.991714:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1132 00002000:00100000:1.0:1788845048.991721:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03beab100 x1875739641858560/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845048.991729:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03beab100 with x1875739641858560 00000100:00100000:0.0:1788845048.995718:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845048.995818:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641858688 00000100:00100000:3.0:1788845048.995827:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641858688 00002000:00100000:3.0:1788845048.995875:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5f80 x1875739641858688/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/0 e 0 to 0 dl 1788845059 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.995917:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5f80 with x1875739641858688 00000100:00100000:3.0:1788845048.995953:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1133 00000100:00100000:3.0:1788845048.995964:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641858688:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845048.996210:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641858688:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 247us (501us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845048.996217:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1133 00002000:00100000:3.0:1788845048.996224:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc5f80 x1875739641858688/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:154/0 lens 488/4536 e 0 to 0 dl 1788845059 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845048.996231:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc5f80 with x1875739641858688 00000100:00100000:0.0:1788845049.020005:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.020048:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.020140:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641858816 00000100:00100000:1.0:1788845049.020151:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641858816 00000100:00100000:1.0:1788845049.020228:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1680 00000100:00100000:1.0:1788845049.020240:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2368:x1875739641858816:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:1.0:1788845049.020263:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845049.020282:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8c4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.020320:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845049.020332:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:1.0:1788845049.020341:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.020355:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845049.020358:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845049.020363:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1200, rc 0) 00000100:00100000:1.0:1788845049.020414:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2368:x1875739641858816:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 177us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845049.020421:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1680 00000100:00100000:2.0:1788845049.020672:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641858944 00000100:00100000:2.0:1788845049.020678:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641858944 00000100:00100000:2.0:1788845049.020719:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1272 00000100:00100000:2.0:1788845049.020727:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641858944:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845049.020736:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845049.020747:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af4a6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8cb expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845049.020808:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660429184:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845049.020908:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4ccc0 going to sleep for 11 seconds 00000100:00080000:3.0:1788845049.021143:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788845049.021148:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845049.049040:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660429184:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845049.049081:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af4a6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8cb expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.049106:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af4a6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8cb expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.049120:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845049.049245:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641858944:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 28518us (29195us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845049.049265:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1272 00000100:00100000:1.0:1788845049.055668:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.056243:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641859072 00000100:00100000:0.0:1788845049.056317:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641859072 00000100:00100000:0.0:1788845049.056451:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1273 00000100:00100000:0.0:1788845049.056463:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641859072:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845049.056483:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845049.056502:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af4ad lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8d2 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845049.056576:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660429312:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845049.056665:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18900 going to sleep for 11 seconds 00000100:00100000:0.0:1788845049.066153:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01fc4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660429312:192.168.201.29@tcp:106: 00010000:00010000:0.0:1788845049.066195:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af4ad lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d2 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.066223:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11db578/0xb2187301287af4ad lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d2 expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.066237:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845049.067199:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641859072:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 10734us (11536us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845049.067209:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1273 00000100:00100000:2.0:1788845049.071049:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.071168:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641859200 00000100:00100000:1.0:1788845049.071177:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641859200 00002000:00100000:1.0:1788845049.071275:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea8e00 x1875739641859200/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845049.071283:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea8e00 with x1875739641859200 00000100:00100000:1.0:1788845049.071314:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1134 00000100:00100000:1.0:1788845049.071324:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641859200:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845049.071531:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641859200:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 207us (490us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845049.071539:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1134 00002000:00100000:1.0:1788845049.071545:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03bea8e00 x1875739641859200/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845049.071552:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03bea8e00 with x1875739641859200 00000100:00080000:3.0:1788845049.071859:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845049.075430:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.075624:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641859328 00000100:00100000:3.0:1788845049.075633:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641859328 00002000:00100000:3.0:1788845049.075680:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90a80 x1875739641859328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.075691:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90a80 with x1875739641859328 00000100:00100000:3.0:1788845049.075772:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1135 00000100:00100000:3.0:1788845049.075782:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641859328:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845049.076029:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641859328:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 247us (616us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845049.076037:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1135 00002000:00100000:3.0:1788845049.076044:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a019d90a80 x1875739641859328/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.076052:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a019d90a80 with x1875739641859328 00000100:00100000:0.0:1788845049.079531:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.079639:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641859456 00000100:00100000:2.0:1788845049.079649:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641859456 00002000:00100000:2.0:1788845049.079693:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc4700 x1875739641859456/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.079702:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4700 with x1875739641859456 00000100:00100000:2.0:1788845049.079733:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1136 00000100:00100000:2.0:1788845049.079740:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641859456:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845049.079963:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641859456:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 222us (440us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845049.079967:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1136 00002000:00100000:2.0:1788845049.079972:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc4700 x1875739641859456/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.079977:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc4700 with x1875739641859456 00000100:00100000:0.0:1788845049.083397:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.083684:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641859584 00000100:00100000:2.0:1788845049.083694:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641859584 00002000:00100000:2.0:1788845049.083740:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6d80 x1875739641859584/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.083750:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6d80 with x1875739641859584 00000100:00100000:2.0:1788845049.083833:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1137 00000100:00100000:2.0:1788845049.083843:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641859584:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845049.084119:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641859584:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 276us (731us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845049.084127:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1137 00002000:00100000:2.0:1788845049.084134:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f01fc6d80 x1875739641859584/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.084143:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f01fc6d80 with x1875739641859584 00000100:00100000:0.0:1788845049.090505:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.090909:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641859712 00000100:00100000:2.0:1788845049.090918:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641859712 00002000:00100000:2.0:1788845049.090966:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532a680 x1875739641859712/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.090974:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532a680 with x1875739641859712 00000100:00100000:2.0:1788845049.091002:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1138 00000100:00100000:2.0:1788845049.091010:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641859712:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845049.091236:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641859712:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 226us (741us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845049.091244:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1138 00002000:00100000:2.0:1788845049.091251:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532a680 x1875739641859712/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.091259:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532a680 with x1875739641859712 00000100:00100000:0.0:1788845049.094705:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.094793:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641859840 00000100:00100000:3.0:1788845049.094803:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641859840 00002000:00100000:3.0:1788845049.094851:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532b100 x1875739641859840/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.094861:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532b100 with x1875739641859840 00000100:00100000:3.0:1788845049.094930:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1139 00000100:00100000:3.0:1788845049.094940:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641859840:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845049.095169:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641859840:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 231us (474us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845049.095177:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1139 00002000:00100000:3.0:1788845049.095184:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532b100 x1875739641859840/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.095192:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532b100 with x1875739641859840 00000100:00100000:0.0:1788845049.098325:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.098430:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641859968 00000100:00100000:2.0:1788845049.098441:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641859968 00002000:00100000:2.0:1788845049.098501:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035328e00 x1875739641859968/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.098513:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328e00 with x1875739641859968 00000100:00100000:2.0:1788845049.098552:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1140 00000100:00100000:2.0:1788845049.098562:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641859968:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845049.098827:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035328e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641859968:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 266us (512us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845049.098838:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1140 00002000:00100000:2.0:1788845049.098846:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a035328e00 x1875739641859968/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.098854:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035328e00 with x1875739641859968 00000100:00100000:0.0:1788845049.101926:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.102053:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641860096 00000100:00100000:3.0:1788845049.102064:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641860096 00002000:00100000:3.0:1788845049.102126:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532bb80 x1875739641860096/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.102136:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532bb80 with x1875739641860096 00000100:00100000:3.0:1788845049.102176:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1141 00000100:00100000:3.0:1788845049.102186:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03532bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641860096:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845049.102449:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03532bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2370:x1875739641860096:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 258us (562us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845049.102457:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1141 00002000:00100000:3.0:1788845049.102465:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03532bb80 x1875739641860096/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.102473:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03532bb80 with x1875739641860096 00000100:00100000:0.0:1788845049.107182:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.107285:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641860224 00000100:00100000:2.0:1788845049.107296:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641860224 00002000:00100000:2.0:1788845049.107346:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c1500 x1875739641860224/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.107358:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c1500 with x1875739641860224 00000100:00100000:2.0:1788845049.107396:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1142 00000100:00100000:2.0:1788845049.107405:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641860224:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845049.107686:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641860224:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 281us (514us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845049.107693:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1142 00002000:00100000:2.0:1788845049.107700:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c1500 x1875739641860224/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.107707:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c1500 with x1875739641860224 00000100:00100000:0.0:1788845049.111672:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.111759:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641860352 00000100:00100000:3.0:1788845049.111769:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641860352 00002000:00100000:3.0:1788845049.111819:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c2a00 x1875739641860352/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.111831:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c2a00 with x1875739641860352 00000100:00100000:3.0:1788845049.111870:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1143 00000100:00100000:3.0:1788845049.111880:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641860352:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845049.112167:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641860352:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 288us (503us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845049.112176:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1143 00002000:00100000:3.0:1788845049.112182:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a03f6c2a00 x1875739641860352/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.112189:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03f6c2a00 with x1875739641860352 00000100:00100000:0.0:1788845049.115831:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.115954:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641860480 00000100:00100000:1.0:1788845049.115964:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641860480 00002000:00100000:1.0:1788845049.116012:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849880 x1875739641860480/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845049.116023:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849880 with x1875739641860480 00000100:00100000:1.0:1788845049.116064:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1144 00000100:00100000:1.0:1788845049.116076:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641860480:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845049.116329:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641860480:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 255us (509us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845049.116337:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1144 00002000:00100000:1.0:1788845049.116344:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849880 x1875739641860480/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845049.116352:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849880 with x1875739641860480 00000100:00100000:0.0:1788845049.119540:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.119630:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641860608 00000100:00100000:3.0:1788845049.119638:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641860608 00002000:00100000:3.0:1788845049.119687:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848380 x1875739641860608/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.119697:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848380 with x1875739641860608 00000100:00100000:3.0:1788845049.119734:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1145 00000100:00100000:3.0:1788845049.119744:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641860608:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845049.120021:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641860608:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 277us (490us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845049.120029:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1145 00002000:00100000:3.0:1788845049.120036:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03848380 x1875739641860608/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.120050:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03848380 with x1875739641860608 00000100:00100000:0.0:1788845049.123339:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.123455:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641860736 00000100:00100000:1.0:1788845049.123465:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641860736 00002000:00100000:1.0:1788845049.123519:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849500 x1875739641860736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845049.123530:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849500 with x1875739641860736 00000100:00100000:1.0:1788845049.123580:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1146 00000100:00100000:1.0:1788845049.123590:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641860736:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845049.123843:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2368:x1875739641860736:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 254us (514us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845049.123851:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1146 00002000:00100000:1.0:1788845049.123857:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f03849500 x1875739641860736/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845049.123865:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f03849500 with x1875739641860736 00000100:00100000:0.0:1788845049.128493:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.128755:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641860864 00000100:00100000:2.0:1788845049.128763:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641860864 00000100:00100000:2.0:1788845049.128809:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1681 00000100:00100000:2.0:1788845049.128816:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641860864:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845049.128833:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845049.128836:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788845049.128872:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af498 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8bd expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.128915:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845049.128918:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845049.128931:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8d9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.128955:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.128965:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.128995:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.129010:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 12 pid: 11036 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845049.129020:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660429440:192.168.201.29@tcp:104: 00000100:00100000:2.0:1788845049.129078:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845049.131287:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.131346:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641860992 00000100:00100000:3.0:1788845049.131353:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641860992 00010000:00010000:3.0:1788845049.131385:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 13 pid: 11036 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845049.131415:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 571 00000100:00100000:3.0:1788845049.131424:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+13:2369:x1875739641860992:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845049.131437:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845049.131471:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 13 pid: 11036 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845049.131488:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 13 pid: 11036 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845049.131510:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:3.0:1788845049.131516:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845049.131527:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845049.131536:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845049.131581:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845049.131595:0:5824:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660429568:192.168.201.29@tcp:105: 00000100:00100000:3.0:1788845049.131655:0:5824:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22a80 going to sleep for 11 seconds 00010000:00010000:2.0:1788845049.132272:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af49f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8c4 expref: 13 pid: 11036 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845049.132290:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660429440:192.168.201.29@tcp:104: 00010000:00010000:2.0:1788845049.132312:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.132329:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845049.132332:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845049.132335:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865800, rc 0) 00000100:00100000:2.0:1788845049.132373:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641860864:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 3557us (3891us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845049.132380:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1681 00000100:00100000:3.0:1788845049.137370:0:5824:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5824:1875739660429568:192.168.201.29@tcp:105: 00010000:00010000:3.0:1788845049.137397:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845049.137459:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641860992:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 6035us (6177us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845049.137470:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 571 00000100:00100000:1.0:1788845049.140432:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.140489:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.140590:0:6688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641861248 00000100:00100000:0.0:1788845049.140601:0:6688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641861248 00000100:00100000:0.0:1788845049.140688:0:6688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 641 00000100:00100000:0.0:1788845049.140698:0:6688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01ceea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641861248:12345-192.168.201.29@tcp:22:fsx.0 00002000:00010000:0.0:1788845049.141042:0:6688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1790875 -> 1790875 00002000:00010000:0.0:1788845049.141047:0:6688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845048 -> 1788845048 00002000:00010000:0.0:1788845049.141049:0:6688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845048 -> 1788845048 00002000:00010000:0.0:1788845049.141051:0:6688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845048 -> 1788845048 00002000:00010000:0.0:1788845049.141053:0:6688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 179 -> 179 00000100:00100000:0.0:1788845049.141117:0:6688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01ceea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641861248:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 419us (626us total) trans 4294967521 rc 0/0 00000100:00100000:0.0:1788845049.141125:0:6688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 641 00000100:00080000:2.0:1788845049.141292:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845049.141429:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641861120 00000100:00100000:3.0:1788845049.141435:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641861120 00000100:00100000:3.0:1788845049.141479:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1147 00000100:00100000:3.0:1788845049.141486:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:2369:x1875739641861120:12345-192.168.201.29@tcp:22:fsx.0 00000004:00010000:3.0:1788845049.141609:0:13305:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 887101 -> 887101 00000004:00010000:3.0:1788845049.141612:0:13305:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845048 -> 1788845048 00000004:00010000:3.0:1788845049.141615:0:13305:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845047 -> 1788845047 00000004:00010000:3.0:1788845049.141617:0:13305:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845048 -> 1788845048 00000004:00010000:3.0:1788845049.141620:0:13305:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845049.141658:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:2369:x1875739641861120:12345-192.168.201.29@tcp:22:fsx.0 Request processed in 172us (1235us total) trans 4294967884 rc 0/0 00000100:00100000:3.0:1788845049.141664:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1147 00000100:00100000:1.0:1788845049.145183:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.145236:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.145312:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641861376 00000100:00100000:2.0:1788845049.145319:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641861376 00000100:00100000:2.0:1788845049.145369:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1682 00000100:00100000:2.0:1788845049.145377:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641861376:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845049.145395:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845049.145411:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af4bb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8e0 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.145454:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845049.145498:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660429696:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845049.145580:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:0.0:1788845049.145636:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641861504 00000100:00100000:0.0:1788845049.145641:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641861504 00000100:00100000:0.0:1788845049.145679:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1274 00000100:00100000:0.0:1788845049.145686:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641861504:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845049.145695:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845049.145707:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287af4c2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8e7 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845049.145743:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660429824:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845049.145938:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18900 going to sleep for 11 seconds 00002000:00010000:0.0:1788845049.147820:0:6682:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime: 1788845048 -> 1788845049 00002000:00010000:0.0:1788845049.147831:0:6682:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime: 1788845048 -> 1788845049 00000100:00100000:0.0:1788845049.147854:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6682:1875739660429824:192.168.201.29@tcp:106: 00000004:00010000:2.0:1788845049.147904:0:11036:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845047 -> 1788845049 00000004:00010000:2.0:1788845049.147908:0:11036:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788845048 -> 1788845049 00010000:00010000:0.0:1788845049.147917:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287af4c2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8e7 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845049.147925:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03bea9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660429696:192.168.201.29@tcp:106: 00002000:00010000:2.0:1788845049.147937:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:0.0:1788845049.147940:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003dc85a9f/0xb2187301287af4c2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8e7 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.147947:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af4bb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8e0 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.147953:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00010000:00010000:2.0:1788845049.147960:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845049.147963:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845049.147967:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af4bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8e0 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.147977:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864e00, rc 0) 00000100:00100000:0.0:1788845049.148007:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641861504:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2320us (2769us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845049.148015:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1274 00000100:00100000:2.0:1788845049.148071:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01cece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641861376:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2694us (2898us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845049.148077:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1682 00000100:00100000:2.0:1788845049.149724:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.149974:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641861632 00000100:00100000:0.0:1788845049.149982:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641861632 00000100:00100000:2.0:1788845049.150067:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.150113:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1683 00000100:00100000:0.0:1788845049.150123:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641861632:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845049.150145:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845049.150160:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af4c9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8ee expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.150191:0:5840:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845049.150219:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660429952:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845049.150291:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18900 going to sleep for 11 seconds 00000100:00100000:3.0:1788845049.150455:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641861760 00000100:00100000:3.0:1788845049.150460:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641861760 00000100:00100000:3.0:1788845049.150498:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1275 00000100:00100000:3.0:1788845049.150506:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641861760:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:3.0:1788845049.150518:0:6685:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845049.150528:0:6685:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287af4d0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8f5 expref: -99 pid: 6685 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845049.150562:0:6685:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660430080:192.168.201.29@tcp:106: 00000100:00100000:3.0:1788845049.150612:0:6685:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788845049.154812:0:6685:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0360b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6685:1875739660430080:192.168.201.29@tcp:106: 00010000:00010000:3.0:1788845049.154855:0:6685:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287af4d0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8f5 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845049.154913:0:6685:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008576d7d0/0xb2187301287af4d0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8f5 expref: 6 pid: 6685 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845049.154931:0:6685:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216fa00, rc 0) 00000100:00100000:3.0:1788845049.154989:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641861760:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 4483us (4919us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845049.154998:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1275 00000100:00100000:0.0:1788845049.156171:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660429952:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845049.156191:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 887101 00010000:00010000:0.0:1788845049.156203:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af4c9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8ee expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.156215:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845049.156217:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845049.156220:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af4c9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8ee expref: 13 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.156231:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845049.156361:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641861632:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6238us (6643us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845049.156369:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1683 00000100:00100000:2.0:1788845049.160024:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.160136:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641862016 00000100:00100000:1.0:1788845049.160144:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641862016 00000100:00100000:1.0:1788845049.160210:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1684 00000100:00100000:1.0:1788845049.160218:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641862016:12345-192.168.201.29@tcp:36:fsx.0 00010000:00010000:1.0:1788845049.160276:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845049.160298:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af459 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa87e expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.160312:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845049.160315:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845049.160516:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.160530:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.160535:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.160561:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 11 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.160587:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 11 pid: 5839 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845049.160605:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023ddc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660430208:192.168.201.29@tcp:104: 00000100:00100000:1.0:1788845049.160729:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01bf4cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845049.162989:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.163078:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641862144 00000100:00100000:0.0:1788845049.163092:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641862144 00010000:00010000:0.0:1788845049.163122:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 12 pid: 5839 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845049.163154:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 572 00000100:00100000:0.0:1788845049.163161:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082ff850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641862144:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845049.163171:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845049.163176:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 12 pid: 5839 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.163189:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 12 pid: 5839 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.163206:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845049.163210:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.163218:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.163223:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.163235:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845049.163274:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082ff850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2369:x1875739641862144:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 112us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845049.163281:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 572 00010000:00010000:1.0:1788845049.163862:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af3db lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa807 expref: 11 pid: 5839 timeout: 538 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845049.163877:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023ddc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660430208:192.168.201.29@tcp:104: 00010000:00010000:1.0:1788845049.164189:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.164199:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.164205:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788845049.164323:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967885 00000004:00010000:1.0:1788845049.164370:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 887101 -> 887101 00000004:00010000:1.0:1788845049.164374:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845048 -> 1788845049 00000004:00010000:1.0:1788845049.164376:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845049 -> 1788845047 00000004:00010000:1.0:1788845049.164378:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845049 -> 1788845049 00000004:00010000:1.0:1788845049.164380:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000004:00010000:1.0:1788845049.164387:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000499c7704 reply state 00000000546df1d6 transno 4294967886 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845049.164395:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845049.164593:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641862016:12345-192.168.201.29@tcp:36:fsx.0 Request processed in 4374us (4575us total) trans 4294967886 rc 0/0 00000100:00100000:1.0:1788845049.164600:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1684 00000020:00080000:1.0:1788845049.164605:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788845049.167143:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.167160:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.167168:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.167175:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c0c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845049.167180:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.167188:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.167206:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:1.0:1788845049.167209:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845049.167213:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845049.167218:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0c00/0xb2187301287af4d7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845049.167711:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.167751:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.167752:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641862272 00000100:00100000:1.0:1788845049.167760:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641862272 00002000:00010000:1.0:1788845049.167795:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845049.167799:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a00891bb80 with x1875739641862272 00000100:00100000:0.0:1788845049.167858:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641862400 00000100:00100000:0.0:1788845049.167863:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641862400 00002000:00010000:0.0:1788845049.167914:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845049.167919:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a008918a80 with x1875739641862400 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845049.167929:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a008918a80. 00000100:00100000:0.0:1788845049.167955:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 642 00000100:00100000:0.0:1788845049.167965:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2369:x1875739641862400:12345-192.168.201.29@tcp:10:fsx.0 00002000:00010000:0.0:1788845049.168303:0:6690:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 1790875 -> 0 00002000:00010000:0.0:1788845049.168307:0:6690:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1788845048 -> 1788845049 00002000:00010000:0.0:1788845049.168310:0:6690:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1788845049 -> 1788845048 00002000:00010000:0.0:1788845049.168313:0:6690:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1788845049 -> 1788845049 00002000:00010000:0.0:1788845049.168315:0:6690:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 179 -> 179 00000100:00100000:0.0:1788845049.168376:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641862400:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 412us (623us total) trans 4294967522 rc 0/0 00000100:00100000:0.0:1788845049.168384:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 642 00002000:00010000:0.0:1788845049.168388:0:6690:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:0.0:1788845049.168391:0:6690:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff94a008918a80 with x1875739641862400 ext(0->18446744073709551615) 00002000:00010000:0.0:1788845049.168396:0:6690:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff94a008918a80. 00000100:00100000:1.0:1788845049.169037:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1148 00000100:00100000:1.0:1788845049.169049:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641862272:12345-192.168.201.29@tcp:10:fsx.0 00000004:00010000:1.0:1788845049.169195:0:5853:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 887101 -> 916310 00000004:00010000:1.0:1788845049.169199:0:5853:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845049 -> 1788845049 00000004:00010000:1.0:1788845049.169202:0:5853:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845047 -> 1788845047 00000004:00010000:1.0:1788845049.169204:0:5853:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845049 -> 1788845049 00000004:00010000:1.0:1788845049.169207:0:5853:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:1.0:1788845049.169259:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:2369:x1875739641862272:12345-192.168.201.29@tcp:10:fsx.0 Request processed in 213us (1551us total) trans 4294967887 rc 0/0 00000100:00100000:1.0:1788845049.169266:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1148 00002000:00010000:1.0:1788845049.169270:0:5853:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788845049.169273:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a00891bb80 with x1875739641862272 00000100:00100000:2.0:1788845049.175509:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.175594:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641862528 00000100:00100000:1.0:1788845049.175604:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641862528 00000100:00100000:0.0:1788845049.175802:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1685 00000100:00100000:0.0:1788845049.175815:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+20:14891:x1875739641862528:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845049.175843:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845049.175864:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af4de lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa8fc expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.175983:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287af4e5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.176000:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287af4e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.176012:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287af4e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845049.176040:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 916310 00000004:00200000:0.0:1788845049.176045:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845049.176066:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287af4e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.176107:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af4de lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8fc expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.176125:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007a00/0xb2187301287af4e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8fc expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.176135:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845049.176139:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845049.176142:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007a00, rc 0) 00000100:00100000:0.0:1788845049.176421:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:14891:x1875739641862528:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 607us (921us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845049.176429:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1685 00000100:00100000:0.0:1788845049.182996:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.184662:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641862656 00000100:00100000:2.0:1788845049.184677:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641862656 00000100:00100000:2.0:1788845049.184742:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1686 00000100:00100000:2.0:1788845049.184752:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641862656:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845049.184774:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845049.184794:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af4ec lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa903 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.184879:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af4f3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.184929:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af4f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.184939:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af4f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845049.184964:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 916310 00000004:00200000:2.0:1788845049.184968:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845049.184985:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af4f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.185010:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af4ec lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa903 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.185027:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af4f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa903 expref: 12 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.185035:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:2.0:1788845049.185038:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845049.185041:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864800, rc 0) 00000100:00100000:2.0:1788845049.185117:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:14891:x1875739641862656:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 366us (2132us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845049.185124:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1686 00000100:00100000:1.0:1788845049.188081:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.188129:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.188203:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641862784 00000100:00100000:0.0:1788845049.188212:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641862784 00000100:00100000:0.0:1788845049.188270:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1687 00000100:00100000:0.0:1788845049.188279:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641862784:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845049.188299:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845049.188318:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af4fa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa90a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.188351:0:5841:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845049.188387:0:5841:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660430336:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845049.188470:0:5841:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845049.188844:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641862912 00000100:00100000:2.0:1788845049.188849:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641862912 00000100:00100000:2.0:1788845049.188880:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1276 00000100:00100000:2.0:1788845049.188911:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641862912:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845049.188921:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845049.188931:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af501 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa911 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845049.188966:0:7983:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660430464:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845049.189002:0:7983:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000004:00010000:0.0:1788845049.195021:0:5841:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788845047 -> 1788845048 00000100:00100000:0.0:1788845049.195049:0:5841:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5841:1875739660430336:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845049.195070:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 916310 00010000:00010000:0.0:1788845049.195090:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af4fa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa90a expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.195111:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845049.195114:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845049.195120:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af4fa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa90a expref: 13 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.195135:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006200, rc 0) 00000100:00100000:0.0:1788845049.195198:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2370:x1875739641862784:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6919us (7128us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845049.195208:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1687 00000100:00100000:2.0:1788845049.195406:0:7983:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:7983:1875739660430464:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845049.195422:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af501 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa911 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.195434:0:7983:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b5e992f9/0xb2187301287af501 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa911 expref: 6 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.195440:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:2.0:1788845049.195472:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2370:x1875739641862912:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 6563us (7343us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845049.195477:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1276 00000100:00100000:1.0:1788845049.201043:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.201116:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.201186:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641863168 00000100:00100000:0.0:1788845049.201197:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641863168 00000100:00100000:0.0:1788845049.201257:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1277 00000100:00100000:0.0:1788845049.201269:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641863168:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:0.0:1788845049.201288:0:6684:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845049.201306:0:6684:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004f9c76d6/0xb2187301287af508 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa91f expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845049.201355:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660430592:192.168.201.29@tcp:106: 00000100:00100000:0.0:1788845049.201422:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008c18e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788845049.201971:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641863040 00000100:00100000:2.0:1788845049.201977:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641863040 00000100:00100000:2.0:1788845049.202057:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1688 00000100:00100000:2.0:1788845049.202066:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641863040:12345-192.168.201.29@tcp:101:fsx.0 00010000:00010000:2.0:1788845049.202080:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845049.202096:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af50f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa918 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.202126:0:11036:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865800/0xb2187301287af4b4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa8d9 expref: 20 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845049.202163:0:11036:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660430720:192.168.201.29@tcp:106: 00000100:00100000:2.0:1788845049.202222:0:11036:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a019f4c540 going to sleep for 11 seconds 00000100:00100000:2.0:1788845049.203306:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660430592:192.168.201.29@tcp:106: 00010000:00010000:2.0:1788845049.203340:0:6684:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004f9c76d6/0xb2187301287af508 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa91f expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.203364:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f9c76d6/0xb2187301287af508 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa91f expref: 6 pid: 6684 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845049.203377:0:6684:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006200, rc 0) 00000100:00100000:2.0:1788845049.203427:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641863168:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2159us (2309us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845049.203435:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1277 00000100:00100000:0.0:1788845049.204494:0:11036:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891a300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:11036:1875739660430720:192.168.201.29@tcp:106: 00002000:00010000:0.0:1788845049.204514:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 916310 00010000:00010000:0.0:1788845049.204529:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af50f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa918 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.204546:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845049.204549:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845049.204554:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865a00/0xb2187301287af50f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa918 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845049.204577:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865a00, rc 0) 00000100:00100000:0.0:1788845049.204623:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+12:2367:x1875739641863040:12345-192.168.201.29@tcp:101:fsx.0 Request processed in 2557us (3587us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845049.204631:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1688 00000100:00100000:3.0:1788845049.207769:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.208786:0:13305:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641863296 00000100:00100000:2.0:1788845049.208796:0:13305:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641863296 00002000:00100000:2.0:1788845049.208842:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b6300 x1875739641863296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.208851:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6300 with x1875739641863296 00000100:00100000:2.0:1788845049.208933:0:13305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1149 00000100:00100000:2.0:1788845049.208944:0:13305:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641863296:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845049.209181:0:13305:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641863296:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 238us (1421us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845049.209189:0:13305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1149 00002000:00100000:2.0:1788845049.209197:0:13305:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b6300 x1875739641863296/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.209205:0:13305:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b6300 with x1875739641863296 00000100:00100000:3.0:1788845049.211587:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845049.211791:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641863424 00000100:00100000:2.0:1788845049.211801:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641863424 00002000:00100000:2.0:1788845049.211848:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5f80 x1875739641863424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.211858:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5f80 with x1875739641863424 00000100:00100000:2.0:1788845049.212184:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1150 00000100:00100000:2.0:1788845049.212195:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641863424:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:2.0:1788845049.212447:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641863424:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 253us (868us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788845049.212455:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1150 00002000:00100000:2.0:1788845049.212462:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b5f80 x1875739641863424/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845049.212470:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b5f80 with x1875739641863424 00000100:00100000:1.0:1788845049.215689:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.215814:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641863552 00000100:00100000:3.0:1788845049.215824:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641863552 00002000:00100000:3.0:1788845049.215866:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dce00 x1875739641863552/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.215874:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dce00 with x1875739641863552 00000100:00100000:3.0:1788845049.215936:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1151 00000100:00100000:3.0:1788845049.215947:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641863552:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845049.216184:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641863552:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 239us (507us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845049.216192:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1151 00002000:00100000:3.0:1788845049.216198:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f023dce00 x1875739641863552/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.216206:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f023dce00 with x1875739641863552 00000100:00100000:3.0:1788845049.218473:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.218541:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641863680 00000100:00100000:1.0:1788845049.218551:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641863680 00002000:00100000:1.0:1788845049.218613:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b7800 x1875739641863680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845049.218624:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7800 with x1875739641863680 00000100:00100000:1.0:1788845049.218660:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1152 00000100:00100000:1.0:1788845049.218670:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641863680:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:1.0:1788845049.218877:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:2368:x1875739641863680:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 208us (410us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788845049.218908:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1152 00002000:00100000:1.0:1788845049.218914:0:5853:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a0360b7800 x1875739641863680/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845049.218922:0:5853:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a0360b7800 with x1875739641863680 00000100:00100000:0.0:1788845049.224364:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.224437:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641863808 00000100:00100000:3.0:1788845049.224447:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641863808 00002000:00100000:3.0:1788845049.224497:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384b100 x1875739641863808/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.224510:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384b100 with x1875739641863808 00000100:00100000:3.0:1788845049.224543:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1153 00000100:00100000:3.0:1788845049.224552:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641863808:12345-192.168.201.29@tcp:3:fsx.0 00000100:00100000:3.0:1788845049.224748:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641863808:12345-192.168.201.29@tcp:3:fsx.0 Request processed in 196us (394us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788845049.224756:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1153 00002000:00100000:3.0:1788845049.224763:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff949f0384b100 x1875739641863808/t0(0) o3->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/4536 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.224770:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff949f0384b100 with x1875739641863808 00000100:00100000:2.0:1788845049.229255:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.229341:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641863936 00000100:00100000:3.0:1788845049.229350:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641863936 00002000:00100000:3.0:1788845049.229395:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919500 x1875739641863936/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/0 e 0 to 0 dl 1788845060 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.229412:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919500 with x1875739641863936 00000100:00100000:3.0:1788845049.229455:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1154 00000100:00100000:3.0:1788845049.229464:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008919500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2367:x1875739641863936:12345-192.168.201.29@tcp:4:fsx.0 00000004:00010000:3.0:1788845049.231578:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 916310 -> 916310 00000004:00010000:3.0:1788845049.231588:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788845049 -> 1788845049 00000004:00010000:3.0:1788845049.231591:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788845048 -> 1788845047 00000004:00010000:3.0:1788845049.231593:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788845049 -> 1788845049 00000004:00010000:3.0:1788845049.231595:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 326 -> 326 00000100:00100000:3.0:1788845049.232986:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008919500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+23:2367:x1875739641863936:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 3515us (3731us total) trans 4294967888 rc 0/0 00000100:00100000:3.0:1788845049.232999:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1154 00002000:00100000:3.0:1788845049.233012:0:5852:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff94a008919500 x1875739641863936/t4294967888(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:155/0 lens 488/448 e 0 to 0 dl 1788845060 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788845049.233022:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a008919500 with x1875739641863936 00000100:00100000:1.0:1788845049.238479:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845049.238552:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641864064 00000100:00100000:3.0:1788845049.238573:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641864064 00000100:00100000:3.0:1788845049.238631:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1689 00000100:00100000:3.0:1788845049.238642:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+23:2368:x1875739641864064:12345-192.168.201.29@tcp:16:fsx.0 00000001:00080000:1.0:1788845049.264122:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967887 is committed 00000001:00080000:1.0:1788845049.264145:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967888 is committed 00000001:00080000:1.0:1788845049.264155:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967883 is committed 00000100:00100000:3.0:1788845049.264385:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641864064:12345-192.168.201.29@tcp:16:fsx.0 Request processed in 25738us (25910us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845049.264397:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1689 00000100:00100000:1.0:1788845049.268125:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845049.268330:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641864320 00000100:00100000:0.0:1788845049.268341:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641864320 00000100:00100000:0.0:1788845049.268506:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 40 00000100:00100000:0.0:1788845049.268519:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:14891:x1875739641864320:12345-192.168.201.29@tcp:35:fsx.0 00000100:00100000:0.0:1788845049.268976:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641864320:12345-192.168.201.29@tcp:35:fsx.0 Request processed in 457us (858us total) trans 4294967889 rc 0/0 00000100:00100000:0.0:1788845049.268984:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 40 00000100:00100000:3.0:1788845049.271063:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.271744:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641864576 00000100:00100000:1.0:1788845049.271754:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641864576 00000100:00100000:1.0:1788845049.271806:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 41 00000100:00100000:1.0:1788845049.271818:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:14891:x1875739641864576:12345-192.168.201.29@tcp:35:fsx.0 00000100:00100000:1.0:1788845049.272074:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:14891:x1875739641864576:12345-192.168.201.29@tcp:35:fsx.0 Request processed in 258us (1016us total) trans 4294967890 rc 0/0 00000100:00100000:1.0:1788845049.272144:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 41 00000100:00100000:3.0:1788845049.278271:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.278357:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641864832 00000100:00100000:1.0:1788845049.278367:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641864832 00000100:00100000:1.0:1788845049.278419:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 42 00000100:00100000:1.0:1788845049.278428:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:14891:x1875739641864832:12345-192.168.201.29@tcp:35:fsx.0 00000100:00100000:1.0:1788845049.278747:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641864832:12345-192.168.201.29@tcp:35:fsx.0 Request processed in 320us (495us total) trans 4294967891 rc 0/0 00000100:00100000:1.0:1788845049.278755:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 42 00000100:00100000:2.0:1788845049.284315:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845049.284666:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641865088 00000100:00100000:1.0:1788845049.284676:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641865088 00000100:00100000:1.0:1788845049.284737:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 43 00000100:00100000:1.0:1788845049.284748:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:14891:x1875739641865088:12345-192.168.201.29@tcp:35:fsx.0 00000100:00100000:1.0:1788845049.285044:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:14891:x1875739641865088:12345-192.168.201.29@tcp:35:fsx.0 Request processed in 297us (738us total) trans 4294967892 rc 0/0 00000100:00100000:1.0:1788845049.285051:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 43 00010000:00010000:1.0:1788845049.426138:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845050.451513:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845051.092896:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03849f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660430848:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.092970:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845051.093014:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660431360:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.093037:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845051.094514:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788845051.094540:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660431232:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.094607:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845051.094654:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03848a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660431104:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.094677:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845051.094693:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788845051.094698:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660430976:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.094716:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845051.094728:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660431488:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.094743:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845051.094777:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660430848 00000100:00100000:0.0:1788845051.094784:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660430848 00000100:00100000:0.0:1788845051.094826:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:0.0:1788845051.094834:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660430848:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.094877:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660430848:12345-0@lo:400:kworker.0 Request processed in 44us (1908us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845051.094898:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:0.0:1788845051.094928:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660431360 00000100:00100000:0.0:1788845051.094931:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660431360 00000100:00100000:0.0:1788845051.094958:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660431104 00000100:00100000:0.0:1788845051.094960:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660431104 00000100:00100000:0.0:1788845051.094971:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660430976 00000100:00100000:0.0:1788845051.094974:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660430976 00000100:00100000:0.0:1788845051.094984:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:0.0:1788845051.094989:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660431360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.095017:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660431360:12345-0@lo:400:kworker.0 Request processed in 27us (1979us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845051.097211:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660431232 00000100:00100000:2.0:1788845051.097225:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660431232 00000100:00100000:2.0:1788845051.097285:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660431488 00000100:00100000:2.0:1788845051.097288:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660431488 00000100:00100000:2.0:1788845051.097314:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1788845051.097323:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660431232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845051.097402:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660431232:12345-0@lo:400:kworker.0 Request processed in 80us (2794us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845051.097424:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1788845051.097442:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1788845051.097449:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036081c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660431488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845051.097486:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036081c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660431488:12345-0@lo:400:kworker.0 Request processed in 38us (2742us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845051.097492:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00080000:2.0:1788845051.098368:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788845051.098374:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1788845051.098376:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845051.098401:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:2.0:1788845051.099655:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0399e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660431104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845051.099737:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0399e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660431104:12345-0@lo:400:kworker.0 Request processed in 91us (5057us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845051.099762:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:2.0:1788845051.100148:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:2.0:1788845051.100168:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:2.0:1788845051.100176:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660430976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845051.103261:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660430976:12345-0@lo:400:kworker.0 Request processed in 3075us (8532us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845051.103275:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:2.0:1788845051.103363:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660431232:0@lo:400:kworker.0 00000100:00100000:2.0:1788845051.103377:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03849f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660430848:0@lo:400:kworker.0 00000100:00100000:2.0:1788845051.103390:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660431360:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.108828:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0384bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660430976:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.108848:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01db8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660431488:0@lo:400:kworker.0 00000100:00100000:0.0:1788845051.108860:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03848a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660431104:0@lo:400:kworker.0 00010000:00010000:1.0:1788845051.478799:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845051.603563:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788845051.603588:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660431616:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788845051.603661:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845051.603806:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660431616 00000100:00100000:3.0:1788845051.603815:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660431616 00000100:00100000:3.0:1788845051.603869:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 87 00000100:00100000:3.0:1788845051.603878:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660431616:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788845051.603932:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 160104448 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788845051.603978:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660431616:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (316us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845051.603985:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 87 00000100:00100000:2.0:1788845051.608365:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660431616:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1788845052.498079:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845052.626332:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010481c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660431744:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845052.626414:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845052.626639:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660431744 00000100:00100000:3.0:1788845052.626649:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660431744 00000100:00100000:3.0:1788845052.626696:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 88 00000100:00100000:3.0:1788845052.626706:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660431744:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788845052.626742:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788845052.626795:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660431744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845052.626802:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 88 00000100:00100000:0.0:1788845052.626973:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010481c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660431744:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845052.699383:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845052.699477:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845052.699513:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845052.699546:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845052.699569:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845052.700183:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641865472 00000100:00100000:0.0:1788845052.700194:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641865472 00000100:00100000:0.0:1788845052.700252:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641865728 00000100:00100000:0.0:1788845052.700255:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641865728 00000100:00100000:0.0:1788845052.700277:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 131 00000100:00100000:0.0:1788845052.700286:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641865472:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845052.700350:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+8:2368:x1875739641865472:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 65us (873us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845052.700358:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 131 00000100:00100000:0.0:1788845052.700372:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 132 00000100:00100000:0.0:1788845052.700378:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641865728:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845052.700399:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2368:x1875739641865728:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 21us (894us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845052.700404:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 132 00000100:00100000:2.0:1788845052.701238:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641865344 00000100:00100000:2.0:1788845052.701248:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641865344 00000100:00100000:2.0:1788845052.701291:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641865600 00000100:00100000:2.0:1788845052.701294:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641865600 00000100:00100000:2.0:1788845052.701316:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 180 00000100:00100000:2.0:1788845052.701324:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f775c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641865344:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845052.701373:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f775c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:2368:x1875739641865344:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 49us (1825us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845052.701379:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 180 00000100:00100000:2.0:1788845052.701393:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 181 00000100:00100000:2.0:1788845052.701399:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f774700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641865600:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845052.701424:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f774700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+11:2368:x1875739641865600:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 25us (1854us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845052.701430:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 181 00000100:00080000:3.0:1788845052.702311:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788845052.702319:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845052.704830:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641865216 00000100:00100000:1.0:1788845052.704845:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641865216 00000100:00100000:1.0:1788845052.704931:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 289 00000100:00100000:1.0:1788845052.704941:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641865216:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845052.705149:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641865216:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 208us (5774us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845052.705158:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 289 00010000:00010000:0.0:1788845053.522797:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1788845054.251610:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967522 is committed 00010000:00010000:1.0:1788845054.546350:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1788845054.706749:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967889 is committed 00000001:00080000:2.0:1788845054.706778:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967892 is committed 00000001:00080000:2.0:1788845054.706789:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967890 is committed 00000100:00100000:0.0:1788845055.267538:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845055.267577:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845055.267610:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641865856 00000100:00100000:0.0:1788845055.267619:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641865856 00002000:00100000:0.0:1788845055.267680:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0x10:0x0] req@ffff94a036082d80 x1875739641865856/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:161/0 lens 488/0 e 0 to 0 dl 1788845066 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788845055.267691:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082d80 with x1875739641865856 00000100:00100000:0.0:1788845055.267722:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1155 00000100:00100000:0.0:1788845055.267734:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641865856:12345-192.168.201.29@tcp:4:fsx.0 00000100:00100000:0.0:1788845055.267873:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641865984 00000100:00100000:0.0:1788845055.267878:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641865984 00000100:00100000:0.0:1788845055.267928:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1690 00000100:00100000:0.0:1788845055.267936:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641865984:12345-192.168.201.29@tcp:16:kworker.0 00000100:00100000:0.0:1788845055.268072:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641865984:12345-192.168.201.29@tcp:16:kworker.0 Request processed in 136us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845055.268080:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1690 00000100:00080000:2.0:1788845055.268401:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00010000:1.0:1788845055.275273:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb size from disk: 57344 -> 81834 00000004:00010000:1.0:1788845055.275283:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb mtime from disk: 1788845037 -> 1788845049 00000004:00010000:1.0:1788845055.275287:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb atime from disk: 1788845003 -> 1788845003 00000004:00010000:1.0:1788845055.275290:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb ctime from disk: 1788845037 -> 1788845049 00000004:00010000:1.0:1788845055.275292:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb blocks from disk: 2 -> 130 00000100:00100000:1.0:1788845055.275377:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036082d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641865856:12345-192.168.201.29@tcp:4:fsx.0 Request processed in 7641us (7841us total) trans 4294967893 rc 0/0 00000100:00100000:1.0:1788845055.275389:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1155 00002000:00100000:1.0:1788845055.275402:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0x10:0x0] req@ffff94a036082d80 x1875739641865856/t4294967893(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:161/0 lens 488/448 e 0 to 0 dl 1788845066 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788845055.275411:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a036082d80 with x1875739641865856 00000001:02000400:2.0:1788845055.527815:0:13524:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-dom test sanity: Run sanity with Data-on-MDT files ========================================================== 01:24:14 (1788845054) 00010000:00010000:0.0:1788845055.572499:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845056.210245:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f825880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660432256:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210321:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845056.210392:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1788845056.210398:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03aa9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660431872:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210428:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845056.210456:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660432384:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210481:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845056.210507:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660432128:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210530:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845056.210549:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788845056.210554:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660432000:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210573:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845056.210586:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660432512:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210601:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845056.210625:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660432384 00000100:00100000:0.0:1788845056.210631:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660432384 00000100:00100000:0.0:1788845056.210674:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660432128 00000100:00100000:0.0:1788845056.210676:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660432128 00000100:00100000:0.0:1788845056.210690:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660432000 00000100:00100000:0.0:1788845056.210692:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660432000 00000100:00100000:0.0:1788845056.210707:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1788845056.210715:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660432384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210769:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01db8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660432384:12345-0@lo:400:kworker.0 Request processed in 55us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845056.210777:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1788845056.210794:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:0.0:1788845056.210800:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660432128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210832:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660432128:12345-0@lo:400:kworker.0 Request processed in 32us (300us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845056.210838:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:0.0:1788845056.210849:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1788845056.210853:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660432000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210879:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660432000:12345-0@lo:400:kworker.0 Request processed in 25us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845056.210909:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1788845056.210950:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660432000:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210960:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660432128:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.210985:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660432384:0@lo:400:kworker.0 00000100:00100000:3.0:1788845056.211052:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660432256 00000100:00100000:3.0:1788845056.211057:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660432256 00000100:00100000:3.0:1788845056.211093:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660432512 00000100:00100000:3.0:1788845056.211095:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660432512 00000100:00100000:3.0:1788845056.211112:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:3.0:1788845056.211118:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660432256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845056.211152:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a036083480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660432256:12345-0@lo:400:kworker.0 Request processed in 34us (832us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845056.211157:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:3.0:1788845056.211166:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:3.0:1788845056.211170:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660432512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845056.211194:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660432512:12345-0@lo:400:kworker.0 Request processed in 23us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845056.211199:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:0.0:1788845056.211361:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660432512:0@lo:400:kworker.0 00000100:00100000:0.0:1788845056.211385:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f825880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660432256:0@lo:400:kworker.0 00000100:00100000:2.0:1788845056.211475:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660431872 00000100:00100000:2.0:1788845056.211479:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660431872 00000100:00100000:2.0:1788845056.211513:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:2.0:1788845056.211521:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660431872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845056.211569:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660431872:12345-0@lo:400:kworker.0 Request processed in 49us (1140us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845056.211576:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00080000:2.0:1788845056.211606:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788845056.211610:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1788845056.211612:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845056.211718:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03aa9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660431872:0@lo:400:kworker.0 00010000:00010000:0.0:1788845056.602346:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845056.722413:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660432640:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788845056.722970:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845056.724624:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660432640 00000100:00100000:3.0:1788845056.724637:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660432640 00000100:00100000:3.0:1788845056.724710:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:3.0:1788845056.724721:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660432640:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788845056.724764:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 160104448 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788845056.724847:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660432640:12345-0@lo:13:osp-pre-1-0.0 Request processed in 128us (1878us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845056.724854:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1788845056.725010:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a008918000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660432640:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1788845057.619222:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845057.749830:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788845057.749856:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660432768:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845057.749959:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845057.749994:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660432768 00000100:00100000:3.0:1788845057.750002:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660432768 00000100:00100000:3.0:1788845057.750049:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 90 00000100:00100000:3.0:1788845057.750059:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660432768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788845057.750098:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788845057.750144:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660432768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (186us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845057.750152:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 90 00000100:00100000:2.0:1788845057.750248:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660432768:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845057.818233:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845057.818285:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845057.818382:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845057.818554:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641866112 00000100:00100000:0.0:1788845057.818565:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641866112 00000100:00100000:0.0:1788845057.818623:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 291 00000100:00100000:0.0:1788845057.818634:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2367:x1875739641866112:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845057.818716:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2367:x1875739641866112:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 84us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845057.818725:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 291 00000100:00100000:2.0:1788845057.820022:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845057.820058:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845057.820128:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641866240 00000100:00100000:3.0:1788845057.820136:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641866240 00000100:00100000:3.0:1788845057.820202:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641866496 00000100:00100000:3.0:1788845057.820205:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641866496 00000100:00100000:3.0:1788845057.820228:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 185 00000100:00100000:3.0:1788845057.820238:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641866240:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845057.820325:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:2367:x1875739641866240:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 89us (307us total) trans 0 rc 0/0 00000100:00080000:2.0:1788845057.820499:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845057.820543:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 186 00000100:00100000:2.0:1788845057.820551:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641866496:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845057.820595:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03848a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2367:x1875739641866496:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 46us (536us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845057.820602:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 186 00000100:00100000:1.0:1788845057.820967:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641866368 00000100:00100000:1.0:1788845057.820973:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641866368 00000100:00100000:1.0:1788845057.821005:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641866624 00000100:00100000:1.0:1788845057.821008:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641866624 00000100:00100000:1.0:1788845057.821027:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 135 00000100:00100000:1.0:1788845057.821034:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641866368:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845057.821066:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641866368:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 32us (2780us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845057.821072:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 135 00000100:00100000:1.0:1788845057.821085:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 136 00000100:00100000:1.0:1788845057.821091:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641866624:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845057.821112:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2367:x1875739641866624:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 22us (2731us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845057.821117:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 136 00000100:00080000:2.0:1788845057.821352:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845057.822159:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 185 00010000:00010000:0.0:1788845058.642683:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845059.666291:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1788845059.905236:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967893 is committed 00010000:00010000:0.0:1788845060.691215:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845061.331384:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f827800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660432896:0@lo:400:kworker.0 00000100:00100000:2.0:1788845061.331496:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845061.331682:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660432896 00000100:00100000:0.0:1788845061.331694:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660432896 00000100:00100000:0.0:1788845061.331761:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:0.0:1788845061.331774:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660432896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845061.335341:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f825880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660433280:0@lo:400:kworker.0 00000100:00100000:3.0:1788845061.335452:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845061.335502:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788845061.335510:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660433408:0@lo:400:kworker.0 00000100:00100000:3.0:1788845061.335539:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845061.335572:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660433024:0@lo:400:kworker.0 00000100:00100000:3.0:1788845061.335599:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845061.335628:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660433536:0@lo:400:kworker.0 00000100:00100000:3.0:1788845061.335650:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845061.335669:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845061.335675:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f824000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660433152:0@lo:400:kworker.0 00000100:00100000:3.0:1788845061.335697:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845061.335736:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660433280 00000100:00100000:3.0:1788845061.335745:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660433280 00000100:00100000:3.0:1788845061.335814:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660433536 00000100:00100000:3.0:1788845061.335818:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660433536 00000100:00100000:3.0:1788845061.335845:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:3.0:1788845061.335855:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660433280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845061.336162:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660432896:12345-0@lo:400:kworker.0 Request processed in 4388us (4664us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845061.336174:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:0.0:1788845061.336213:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660433408 00000100:00100000:0.0:1788845061.336217:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660433408 00000100:00100000:0.0:1788845061.336255:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660433024 00000100:00100000:0.0:1788845061.336258:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660433024 00000100:00100000:0.0:1788845061.336271:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660433152 00000100:00100000:0.0:1788845061.336273:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660433152 00000100:00100000:0.0:1788845061.336287:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:0.0:1788845061.336293:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660433408:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845061.336349:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660433408:12345-0@lo:400:kworker.0 Request processed in 55us (809us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845061.336365:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:0.0:1788845061.336378:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:0.0:1788845061.336384:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660433024:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845061.336415:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02361880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660433024:12345-0@lo:400:kworker.0 Request processed in 31us (816us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845061.336421:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:0.0:1788845061.336431:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:0.0:1788845061.336437:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660433152:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845061.336460:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660433152:12345-0@lo:400:kworker.0 Request processed in 24us (763us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845061.336466:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:0.0:1788845061.336495:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1788845061.336502:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660433536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845061.336534:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660433536:12345-0@lo:400:kworker.0 Request processed in 33us (883us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845061.336543:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1788845061.336592:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660433152:0@lo:400:kworker.0 00000100:00100000:0.0:1788845061.336604:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660433024:0@lo:400:kworker.0 00000100:00100000:0.0:1788845061.336613:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660433536:0@lo:400:kworker.0 00000100:00100000:0.0:1788845061.336649:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f824e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660433408:0@lo:400:kworker.0 00000100:00100000:3.0:1788845061.336819:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660433280:12345-0@lo:400:kworker.0 Request processed in 964us (1366us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845061.336827:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00080000:3.0:1788845061.336846:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788845061.336852:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788845061.336854:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845061.338477:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f825880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660433280:0@lo:400:kworker.0 00000100:00100000:2.0:1788845061.347199:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f827800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660432896:0@lo:400:kworker.0 00010000:00010000:1.0:1788845061.721412:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845061.845935:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845061.845967:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660433664:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845061.846057:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845061.846083:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660433664 00000100:00100000:1.0:1788845061.846094:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660433664 00000100:00100000:1.0:1788845061.846147:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 91 00000100:00100000:1.0:1788845061.846158:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a13800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660433664:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788845061.846209:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 160104448 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788845061.846278:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a13800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660433664:12345-0@lo:13:osp-pre-1-0.0 Request processed in 121us (220us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845061.846286:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 91 00000100:00100000:3.0:1788845061.846726:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660433664:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1788845062.738656:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845062.871986:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660433792:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845062.872095:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845062.872492:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660433792 00000100:00100000:3.0:1788845062.872505:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660433792 00000100:00100000:3.0:1788845062.872577:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 92 00000100:00100000:3.0:1788845062.872587:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660433792:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788845062.872632:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788845062.872692:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660433792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 105us (596us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845062.872698:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 92 00000100:00100000:3.0:1788845062.872752:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660433792:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845062.943089:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845062.943150:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845062.943182:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845062.943223:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845062.943368:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845062.943408:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641866880 00000100:00100000:2.0:1788845062.943421:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641866880 00000100:00100000:2.0:1788845062.943496:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641867136 00000100:00100000:2.0:1788845062.943499:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641867136 00000100:00100000:2.0:1788845062.943529:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 190 00000100:00100000:2.0:1788845062.943540:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641866880:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845062.943649:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641866880:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 110us (425us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845062.943657:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 190 00000100:00100000:2.0:1788845062.943677:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 191 00000100:00100000:2.0:1788845062.943684:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641867136:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845062.943835:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641866752 00000100:00100000:0.0:1788845062.943840:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641866752 00000100:00100000:0.0:1788845062.943875:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 293 00000100:00100000:0.0:1788845062.943910:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641866752:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845062.944067:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641867008 00000100:00100000:3.0:1788845062.944072:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641867008 00000100:00100000:3.0:1788845062.944107:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641867264 00000100:00100000:3.0:1788845062.944111:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641867264 00000100:00100000:3.0:1788845062.944136:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 139 00000100:00100000:3.0:1788845062.944142:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0384aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641867008:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845062.944235:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 140 00000100:00100000:1.0:1788845062.944246:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641867264:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845062.944324:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641867136:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 638us (954us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845062.944340:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 191 00000100:00080000:2.0:1788845062.944364:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788845062.944370:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845062.945240:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641866752:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 1351us (2160us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845062.945251:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 293 00000100:00100000:3.0:1788845062.945454:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0384aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641867008:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 1310us (2302us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845062.945464:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 139 00000100:00100000:1.0:1788845062.948872:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641867264:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 4614us (5674us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845062.949001:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 140 00010000:00010000:1.0:1788845063.762139:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845064.787925:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845065.810175:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845066.452240:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660434048:0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.452349:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845066.452417:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660434560:0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.452449:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845066.452485:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845066.452491:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660434176:0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.452513:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845066.452558:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660434048 00000100:00100000:3.0:1788845066.452567:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660434048 00000100:00100000:3.0:1788845066.452618:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660434176 00000100:00100000:3.0:1788845066.452621:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660434176 00000100:00100000:3.0:1788845066.452654:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1788845066.452664:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660434048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.452725:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660434048:12345-0@lo:400:kworker.0 Request processed in 62us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845066.452733:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1788845066.452768:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1788845066.452774:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660434176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.452802:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660434176:12345-0@lo:400:kworker.0 Request processed in 28us (287us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845066.452808:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1788845066.452827:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660434560 00000100:00100000:3.0:1788845066.452831:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660434560 00000100:00100000:3.0:1788845066.452861:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:3.0:1788845066.452868:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660434560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.452930:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660434560:12345-0@lo:400:kworker.0 Request processed in 63us (480us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845066.452937:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00080000:3.0:1788845066.452964:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788845066.452969:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845066.453010:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660434176:0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.453023:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660434048:0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.453034:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660434560:0@lo:400:kworker.0 00000100:00100000:2.0:1788845066.453123:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660433920:0@lo:400:kworker.0 00000100:00100000:2.0:1788845066.453163:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845066.453198:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f657100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660434432:0@lo:400:kworker.0 00000100:00100000:2.0:1788845066.453223:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845066.453243:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788845066.453249:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660434304:0@lo:400:kworker.0 00000100:00100000:2.0:1788845066.453281:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845066.453311:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660434304 00000100:00100000:2.0:1788845066.453315:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660434304 00000100:00100000:2.0:1788845066.453348:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1788845066.453356:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660434304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845066.453391:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660434304:12345-0@lo:400:kworker.0 Request processed in 36us (109us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845066.453397:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00080000:2.0:1788845066.453417:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845066.453434:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660434304:0@lo:400:kworker.0 00000100:00100000:2.0:1788845066.453451:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660434432 00000100:00100000:2.0:1788845066.453454:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660434432 00000100:00100000:2.0:1788845066.453479:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:2.0:1788845066.453484:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660434432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845066.453515:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c48e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660434432:12345-0@lo:400:kworker.0 Request processed in 30us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845066.453520:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00080000:2.0:1788845066.453531:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845066.453545:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f657100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660434432:0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.453794:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660433920 00000100:00100000:3.0:1788845066.453799:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660433920 00000100:00100000:3.0:1788845066.453840:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:3.0:1788845066.453848:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660433920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845066.453916:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660433920:12345-0@lo:400:kworker.0 Request processed in 69us (752us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845066.453923:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00100000:3.0:1788845066.453964:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660433920:0@lo:400:kworker.0 00010000:00010000:1.0:1788845066.834255:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845066.962505:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01c49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660434688:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788845066.962605:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845066.962653:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660434688 00000100:00100000:2.0:1788845066.962663:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660434688 00000100:00100000:2.0:1788845066.962749:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:2.0:1788845066.962760:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660434688:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845066.962810:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 160104448 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788845066.965141:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660434688:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2369us (2522us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845066.965157:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:1.0:1788845066.965879:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01c49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660434688:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1788845067.860683:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845067.986876:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788845067.987039:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660434816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788845067.987110:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845067.987224:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660434816 00000100:00100000:0.0:1788845067.987232:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660434816 00000100:00100000:0.0:1788845067.987315:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 94 00000100:00100000:0.0:1788845067.987325:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660434816:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845067.987362:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845067.987421:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660434816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845067.987428:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 94 00000100:00100000:0.0:1788845067.988593:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660434816:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845068.057350:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845068.057397:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845068.057425:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845068.057446:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845068.057466:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845068.057493:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641867392 00000100:00100000:0.0:1788845068.057502:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641867392 00000100:00100000:0.0:1788845068.057548:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 295 00000100:00100000:0.0:1788845068.057557:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641867392:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845068.057642:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641867392:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 86us (216us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845068.057650:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 295 00000100:00100000:0.0:1788845068.057682:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641867648 00000100:00100000:0.0:1788845068.057686:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641867648 00000100:00100000:0.0:1788845068.057716:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641867904 00000100:00100000:0.0:1788845068.057719:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641867904 00000100:00100000:0.0:1788845068.057742:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 143 00000100:00100000:0.0:1788845068.057748:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641867648:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845068.057783:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641867648:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 35us (335us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845068.057789:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 143 00000100:00100000:0.0:1788845068.057801:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 144 00000100:00100000:0.0:1788845068.057808:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641867904:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845068.057830:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2369:x1875739641867904:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 23us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845068.057837:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 144 00000100:00100000:3.0:1788845068.058005:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641867520 00000100:00100000:3.0:1788845068.058010:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641867520 00000100:00100000:3.0:1788845068.058041:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641867776 00000100:00100000:3.0:1788845068.058044:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641867776 00000100:00100000:3.0:1788845068.058067:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 195 00000100:00100000:3.0:1788845068.058074:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641867520:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845068.058103:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641867520:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 30us (762us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845068.058111:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 195 00000100:00100000:3.0:1788845068.058122:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 196 00000100:00100000:3.0:1788845068.058128:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641867776:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845068.058150:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641867776:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 22us (753us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845068.058155:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 196 00000100:00080000:2.0:1788845068.058469:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1788845068.058474:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1788845068.887160:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845069.909265:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845070.939691:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845071.570345:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660434944:0@lo:400:kworker.0 00000100:00100000:0.0:1788845071.570416:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845071.571252:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660434944 00000100:00100000:3.0:1788845071.571260:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660434944 00000100:00100000:3.0:1788845071.571310:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1788845071.571320:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660434944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845071.571735:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035740000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660435200:0@lo:400:kworker.0 00000100:00100000:1.0:1788845071.571770:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845071.571802:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788845071.571808:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035742d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660435072:0@lo:400:kworker.0 00000100:00100000:1.0:1788845071.571827:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845071.571844:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660435584:0@lo:400:kworker.0 00000100:00100000:1.0:1788845071.571866:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845071.571952:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660434944:12345-0@lo:400:kworker.0 Request processed in 632us (1533us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845071.571962:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1788845071.572009:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660435328:0@lo:400:kworker.0 00000100:00100000:3.0:1788845071.572033:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845071.572143:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788845071.572148:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660435456:0@lo:400:kworker.0 00000100:00100000:3.0:1788845071.572167:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845071.572203:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660435200 00000100:00100000:3.0:1788845071.572209:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660435200 00000100:00100000:3.0:1788845071.572244:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660435072 00000100:00100000:3.0:1788845071.572246:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660435072 00000100:00100000:3.0:1788845071.572257:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660435456 00000100:00100000:3.0:1788845071.572259:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660435456 00000100:00100000:3.0:1788845071.572271:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:3.0:1788845071.572277:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660435200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845071.572317:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660435200:12345-0@lo:400:kworker.0 Request processed in 39us (546us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845071.572323:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:3.0:1788845071.572335:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1788845071.572341:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660435072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845071.572366:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660435072:12345-0@lo:400:kworker.0 Request processed in 26us (538us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845071.572371:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1788845071.572382:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:3.0:1788845071.572387:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660435456:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845071.572414:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660435456:12345-0@lo:400:kworker.0 Request processed in 27us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845071.572419:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00080000:3.0:1788845071.572440:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845071.572466:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660435456:0@lo:400:kworker.0 00000100:00100000:3.0:1788845071.572490:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035742d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660435072:0@lo:400:kworker.0 00000100:00100000:3.0:1788845071.572498:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035740000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660435200:0@lo:400:kworker.0 00000100:00100000:0.0:1788845071.572801:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660434944:0@lo:400:kworker.0 00000100:00100000:1.0:1788845071.574910:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660435584 00000100:00100000:1.0:1788845071.574925:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660435584 00000100:00100000:1.0:1788845071.575006:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660435328 00000100:00100000:1.0:1788845071.575010:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660435328 00000100:00100000:1.0:1788845071.575037:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:1.0:1788845071.575046:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660435584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845071.575146:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660435584:12345-0@lo:400:kworker.0 Request processed in 102us (3278us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845071.575154:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:1.0:1788845071.575169:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:1.0:1788845071.575175:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660435328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845071.575222:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660435328:12345-0@lo:400:kworker.0 Request processed in 47us (3188us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845071.575228:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00080000:3.0:1788845071.575477:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1788845071.575482:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845071.575539:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660435328:0@lo:400:kworker.0 00000100:00100000:3.0:1788845071.575575:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660435584:0@lo:400:kworker.0 00010000:00010000:1.0:1788845071.955188:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845072.083057:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788845072.083082:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660435712:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845072.083151:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845072.083226:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660435712 00000100:00100000:0.0:1788845072.083233:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660435712 00000100:00100000:0.0:1788845072.083282:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 95 00000100:00100000:0.0:1788845072.083292:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660435712:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788845072.083330:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 160104448 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788845072.083376:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660435712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845072.083384:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 95 00000100:00100000:0.0:1788845072.083433:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660435712:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1788845072.978136:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845073.106126:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788845073.108453:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0251df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660435840:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788845073.108562:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845073.108757:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660435840 00000100:00100000:0.0:1788845073.108765:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660435840 00000100:00100000:0.0:1788845073.108824:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 96 00000100:00100000:0.0:1788845073.108836:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660435840:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845073.108875:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845073.108955:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660435840:12345-0@lo:13:osp-pre-0-0.0 Request processed in 122us (393us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845073.108962:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 96 00000100:00100000:1.0:1788845073.109473:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0251df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660435840:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1788845073.117488:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845073.180041:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845073.180085:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845073.180129:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845073.180174:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845073.180556:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641868288 00000100:00100000:1.0:1788845073.180565:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641868288 00000100:00100000:1.0:1788845073.180640:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 147 00000100:00100000:1.0:1788845073.180652:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641868288:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845073.180736:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01c49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641868288:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 86us (650us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845073.180743:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 147 00000100:00100000:3.0:1788845073.180924:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641868160 00000100:00100000:3.0:1788845073.180929:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641868160 00000100:00100000:3.0:1788845073.180959:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641868416 00000100:00100000:3.0:1788845073.180962:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641868416 00000100:00100000:3.0:1788845073.180985:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 200 00000100:00100000:3.0:1788845073.180992:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641868160:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845073.181027:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641868160:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 35us (897us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845073.181032:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 200 00000100:00100000:3.0:1788845073.181044:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 201 00000100:00100000:3.0:1788845073.181049:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641868416:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845073.181070:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2369:x1875739641868416:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 21us (896us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845073.181075:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 201 00000100:00080000:2.0:1788845073.181975:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1788845073.181982:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845073.182222:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641868032 00000100:00100000:0.0:1788845073.182242:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641868032 00000100:00100000:0.0:1788845073.182286:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 297 00000100:00100000:0.0:1788845073.182294:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641868032:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845073.182355:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641868032:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 62us (2325us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845073.182363:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 297 00000100:00100000:2.0:1788845073.186010:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845073.186141:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641868544 00000100:00100000:0.0:1788845073.186161:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641868544 00000100:00080000:2.0:1788845073.186427:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788845073.186482:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1278 00000100:00100000:0.0:1788845073.186495:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+5:2366:x1875739641868544:12345-192.168.201.29@tcp:9:kworker.0 00000020:00000080:0.0:1788845073.186532:0:6684:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xb2187301287a9eed 00000100:00080000:0.0:1788845073.186543:0:6684:0:(import.c:65:import_set_state_nolock()) ffff94a0358d9800 : changing import state from FULL to CLOSED 00000001:00000004:0.0:1788845073.186589:0:6684:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967522 00000100:00100000:0.0:1788845073.187085:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+2:2366:x1875739641868544:12345-192.168.201.29@tcp:9:kworker.0 Request processed in 591us (1085us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845073.187094:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1278 00010000:00010000:1.0:1788845073.188792:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845074.014419:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1788845074.482618:0:14254:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: - need MDS1_VERSION <= 2.16.61-1-g89cf292a8c2 (34683244 <= 34618625) for LU-18938, skip 360 00000001:00100000:2.0:1788845074.686414:0:7641:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a 00000020:00000080:2.0:1788845074.686422:0:7641:0:(genops.c:885:class_export_put()) final put ffff94a0358df800/5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a 00000020:00000080:3.0:1788845074.686524:0:38:0:(genops.c:844:class_export_destroy()) destroying export ffff94a0358df800/5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a for lustre-OST0001 00000020:00000080:3.0:1788845074.686655:0:38:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff94a0358d9800 for lustre-OST0001 00010000:00010000:1.0:1788845075.030630:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845076.050162:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788845076.210489:0:14305:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: - need MDS1_VERSION < v2_14_55-100-g8a84c7f9c7 (34683244 < 34486116) for LU-14927, skip 0f 00000100:00100000:0.0:1788845076.690639:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660436096:0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.690715:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845076.690795:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660436608:0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.690821:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845076.690850:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788845076.690856:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660436224:0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.690876:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845076.690934:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660436096 00000100:00100000:0.0:1788845076.690942:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660436096 00000100:00100000:0.0:1788845076.690993:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660436224 00000100:00100000:0.0:1788845076.690996:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660436224 00000100:00100000:0.0:1788845076.691020:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1788845076.691028:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660436096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.691079:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0251df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660436096:12345-0@lo:400:kworker.0 Request processed in 51us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845076.691086:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1788845076.691115:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1788845076.691122:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660436224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.691149:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f825880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660436224:12345-0@lo:400:kworker.0 Request processed in 28us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845076.691154:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1788845076.691188:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660436224:0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.691197:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02364a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660436096:0@lo:400:kworker.0 00000100:00100000:1.0:1788845076.691408:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660435968:0@lo:400:kworker.0 00000100:00100000:1.0:1788845076.691440:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845076.691475:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660436480:0@lo:400:kworker.0 00000100:00100000:1.0:1788845076.691508:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845076.691528:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788845076.691535:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660436352:0@lo:400:kworker.0 00000100:00100000:1.0:1788845076.691555:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845076.691582:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660436608 00000100:00100000:1.0:1788845076.691586:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660436608 00000100:00100000:1.0:1788845076.691611:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660436352 00000100:00100000:1.0:1788845076.691613:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660436352 00000100:00100000:1.0:1788845076.691630:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:1.0:1788845076.691635:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660436608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845076.691668:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f827800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660436608:12345-0@lo:400:kworker.0 Request processed in 31us (846us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845076.691674:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:1.0:1788845076.691686:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:1.0:1788845076.691692:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660436352:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845076.691718:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660436352:12345-0@lo:400:kworker.0 Request processed in 27us (162us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845076.691724:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:1.0:1788845076.691751:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660436480 00000100:00100000:1.0:1788845076.691755:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660436480 00000100:00100000:1.0:1788845076.691773:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1788845076.691779:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660436480:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845076.691805:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660436480:12345-0@lo:400:kworker.0 Request processed in 27us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845076.691812:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1788845076.691838:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660436352:0@lo:400:kworker.0 00000100:00100000:1.0:1788845076.691848:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660436480:0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.692073:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660435968 00000100:00100000:0.0:1788845076.692078:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660435968 00000100:00100000:0.0:1788845076.692125:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1788845076.692134:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660435968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.692172:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660435968:12345-0@lo:400:kworker.0 Request processed in 41us (731us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845076.692179:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1788845076.692215:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f6c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660435968:0@lo:400:kworker.0 00000100:00100000:0.0:1788845076.692239:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660436608:0@lo:400:kworker.0 00000100:00080000:2.0:1788845076.692357:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1788845076.692362:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788845076.692364:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788845077.074221:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845077.202222:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788845077.202244:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660436736:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845077.202310:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845077.202418:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660436736 00000100:00100000:2.0:1788845077.202425:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660436736 00000100:00100000:2.0:1788845077.202475:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 97 00000100:00100000:2.0:1788845077.202484:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660436736:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845077.202529:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788845077.202576:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660436736:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845077.202583:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 97 00000100:00100000:2.0:1788845077.202625:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660436736:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1788845078.102353:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788845078.109728:0:14353:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: - need OST1_VERSION < v2_17_50-410-gbd8e7b9555 (34683244 < 34681754) for LU-12550, skip 216 00000100:00100000:0.0:1788845078.230400:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660436864:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845078.230498:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845078.232097:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660436864 00000100:00100000:0.0:1788845078.232112:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660436864 00000100:00100000:0.0:1788845078.232808:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:0.0:1788845078.232824:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660436864:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845078.232909:0:6687:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00080000:1.0:1788845078.233274:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845078.234142:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660436864:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1317us (3637us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845078.234155:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000100:00100000:1.0:1788845078.235500:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f826d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660436864:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845078.312629:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845078.312678:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845078.312714:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845078.312752:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845078.312790:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641868800 00000100:00100000:0.0:1788845078.312803:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641868800 00000100:00100000:0.0:1788845078.312873:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641869056 00000100:00100000:0.0:1788845078.312876:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641869056 00000100:00100000:0.0:1788845078.312927:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 205 00000100:00100000:0.0:1788845078.312940:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641868800:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845078.313043:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641868928 00000100:00100000:1.0:1788845078.313048:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641868928 00000100:00100000:0.0:1788845078.313052:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f826d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641868800:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 116us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845078.313070:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 205 00000100:00100000:0.0:1788845078.313090:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 206 00000100:00100000:1.0:1788845078.313095:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 150 00000100:00100000:0.0:1788845078.313097:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641869056:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845078.313101:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f824000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641868928:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845078.313124:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f6c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2368:x1875739641869056:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 28us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845078.313126:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f824000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641868928:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 26us (448us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845078.313133:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 150 00000100:00100000:0.0:1788845078.313135:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 206 00000100:00080000:0.0:1788845078.313157:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788845078.313162:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845078.313309:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641868672 00000100:00100000:3.0:1788845078.313314:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641868672 00000100:00100000:3.0:1788845078.313345:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 299 00000100:00100000:3.0:1788845078.313352:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a008c4e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641868672:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845078.313396:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a008c4e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641868672:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 44us (783us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845078.313403:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 299 00010000:00010000:1.0:1788845079.122382:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1788845079.803836:0:14401:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: excepting tests: 42a 42c 42b 118c 118d 407 119i 216 817 411a 130b 130c 130d 130e 130f 130g 00010000:00010000:1.0:1788845080.146115:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845081.170357:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788845081.474868:0:14449:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842 51b 51c 51e 834 00000100:00100000:1.0:1788845081.811232:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660437248:0@lo:400:kworker.0 00000100:00100000:1.0:1788845081.811305:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845081.811375:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788845081.811381:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660437120:0@lo:400:kworker.0 00000100:00100000:1.0:1788845081.811402:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845081.811420:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660437632:0@lo:400:kworker.0 00000100:00100000:1.0:1788845081.811443:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845081.811477:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660437248 00000100:00100000:1.0:1788845081.811485:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660437248 00000100:00100000:1.0:1788845081.811538:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660437120 00000100:00100000:1.0:1788845081.811541:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660437120 00000100:00100000:1.0:1788845081.811562:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:1.0:1788845081.811570:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660437248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845081.811621:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660437248:12345-0@lo:400:kworker.0 Request processed in 52us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845081.811628:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:1.0:1788845081.811646:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:1.0:1788845081.811652:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660437120:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845081.811677:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660437120:12345-0@lo:400:kworker.0 Request processed in 26us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845081.811683:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:1.0:1788845081.811705:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660437632 00000100:00100000:1.0:1788845081.811709:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660437632 00000100:00100000:1.0:1788845081.811733:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:1.0:1788845081.811739:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660437632:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845081.811769:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660437632:12345-0@lo:400:kworker.0 Request processed in 30us (324us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845081.811774:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00080000:1.0:1788845081.811790:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788845081.811794:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845081.811832:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660437120:0@lo:400:kworker.0 00000100:00100000:1.0:1788845081.811842:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660437632:0@lo:400:kworker.0 00000100:00100000:1.0:1788845081.811851:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660437248:0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.811968:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660437376:0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.812012:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845081.812045:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1788845081.812051:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035be4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660436992:0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.812078:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845081.812098:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660437504:0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.812118:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845081.812142:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660437376 00000100:00100000:2.0:1788845081.812146:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660437376 00000100:00100000:2.0:1788845081.812180:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1788845081.812186:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660437376:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.812220:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660437376:12345-0@lo:400:kworker.0 Request processed in 35us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845081.812226:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1788845081.812243:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660436992 00000100:00100000:2.0:1788845081.812246:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660436992 00000100:00100000:2.0:1788845081.812269:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00000100:00100000:2.0:1788845081.812274:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660436992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.812301:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660436992:12345-0@lo:400:kworker.0 Request processed in 26us (222us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845081.812306:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00080000:2.0:1788845081.812334:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845081.812348:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035be4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660436992:0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.812359:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660437376:0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.812379:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660437504 00000100:00100000:2.0:1788845081.812382:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660437504 00000100:00100000:2.0:1788845081.812407:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1788845081.812413:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660437504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845081.812443:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660437504:12345-0@lo:400:kworker.0 Request processed in 30us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845081.812449:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00080000:2.0:1788845081.812463:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845081.812477:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660437504:0@lo:400:kworker.0 00010000:00010000:1.0:1788845082.194290:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845082.322167:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845082.322194:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660437760:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845082.322278:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845082.322484:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660437760 00000100:00100000:2.0:1788845082.322493:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660437760 00000100:00100000:2.0:1788845082.322552:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 99 00000100:00100000:2.0:1788845082.322561:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660437760:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845082.322596:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788845082.322646:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660437760:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (367us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845082.322652:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 99 00000100:00100000:3.0:1788845082.322861:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660437760:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1788845083.231987:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788845083.263484:0:14497:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: === sanity: start setup 01:24:41 (1788845081) === 00000100:00100000:0.0:1788845083.346127:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660437888:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845083.346199:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845083.346270:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660437888 00000100:00100000:0.0:1788845083.346278:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660437888 00000100:00100000:0.0:1788845083.346334:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 100 00000100:00100000:0.0:1788845083.346343:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660437888:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845083.346381:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845083.346431:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02364a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660437888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (231us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845083.346438:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 100 00000100:00080000:0.0:1788845083.346459:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845083.346498:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02366680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660437888:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845083.418742:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845083.420122:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641869184 00000100:00100000:2.0:1788845083.420135:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641869184 00000100:00100000:2.0:1788845083.420210:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 301 00000100:00100000:2.0:1788845083.420222:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641869184:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845083.420302:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2369:x1875739641869184:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 82us (1568us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845083.420314:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 301 00000100:00100000:0.0:1788845083.425805:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845083.428594:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845083.433177:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845083.434961:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641869312 00000100:00100000:1.0:1788845083.434984:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641869312 00000100:00100000:1.0:1788845083.435368:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641869440 00000100:00100000:1.0:1788845083.435372:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641869440 00000100:00100000:1.0:1788845083.435456:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 153 00000100:00100000:1.0:1788845083.435468:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641869440:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845083.435571:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641869440:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 105us (2405us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845083.435579:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 153 00000100:00100000:1.0:1788845083.435637:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641869568 00000100:00100000:1.0:1788845083.435641:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641869568 00000100:00100000:1.0:1788845083.435664:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 210 00000100:00100000:1.0:1788845083.435671:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641869312:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845083.435702:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02366680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641869312:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 32us (9909us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845083.435709:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 210 00000100:00100000:1.0:1788845083.435721:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 211 00000100:00100000:1.0:1788845083.435728:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641869568:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845083.435754:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2370:x1875739641869568:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 26us (7299us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845083.435759:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 211 00000100:00080000:3.0:1788845083.436398:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788845083.436404:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1788845084.242277:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845085.266386:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845085.759650:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845085.759691:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845085.759790:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641869696 00000100:00100000:1.0:1788845085.759799:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641869696 00000100:00100000:1.0:1788845085.759864:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641869824 00000100:00100000:1.0:1788845085.759867:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641869824 00000100:00100000:1.0:1788845085.759916:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 8 00000100:00100000:1.0:1788845085.759926:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+14:2368:x1875739641869696:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845085.759951:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845085.759968:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff94a031d3ea00/0xb2187301287a9ab0 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f36 expref: 14 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845085.759983:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845085.759985:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 9 00000100:00100000:2.0:1788845085.759999:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641869824:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788845085.760010:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845085.760023:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff94a031d3ee00/0xb2187301287a9ab7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f60 expref: 13 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845085.760030:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02367b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+13:2368:x1875739641869696:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 104us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845085.760038:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 8 00010000:00010000:2.0:1788845085.760039:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788845085.760069:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+11:2368:x1875739641869824:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 74us (377us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845085.760078:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 9 00000100:00100000:1.0:1788845085.762329:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845085.762398:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641869952 00000100:00100000:0.0:1788845085.762407:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641869952 00000100:00100000:0.0:1788845085.762461:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 10 00000100:00100000:0.0:1788845085.762471:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+11:2367:x1875739641869952:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845085.762484:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845085.762500:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff94a031d3ec00/0xb2187301287a9ae1 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f91 expref: 11 pid: 5833 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845085.762516:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845085.762569:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2367:x1875739641869952:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 99us (249us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845085.762578:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 10 00000100:00100000:1.0:1788845085.767586:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845085.767762:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641870464 00000100:00100000:2.0:1788845085.767772:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641870464 00000100:00100000:2.0:1788845085.767918:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1691 00000100:00100000:2.0:1788845085.767930:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+10:2366:x1875739641870464:12345-192.168.201.29@tcp:39:umount.0 00000020:00000080:2.0:1788845085.767977:0:5839:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xb2187301287a9ac5 00000020:00010000:2.0:1788845085.767993:0:5839:0:(genops.c:1921:obd_stale_export_put()) Put export ffff94a0090c5800: total 0 00000100:00080000:2.0:1788845085.767998:0:5839:0:(import.c:65:import_set_state_nolock()) ffff94a0090c5000 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1788845085.768026:0:5839:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 1, last_transno is 4294967893 00000100:00100000:2.0:1788845085.768441:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a+9:2366:x1875739641870464:12345-192.168.201.29@tcp:39:umount.0 Request processed in 512us (862us total) trans 4294967894 rc 0/0 00000100:00100000:2.0:1788845085.768460:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1691 00000020:00010000:0.0:1788845085.769082:0:5828:0:(genops.c:1907:obd_stale_export_get()) Get export ffff94a0090c5800: total 0 00010000:00010000:0.0:1788845085.769129:0:5828:0:(ldlm_lock.c:2651:ldlm_export_cancel_blocked_locks()) Export ffff94a0090c5800, canceled 0 locks, left on hash table 5. 00000004:00010000:0.0:1788845085.769175:0:5828:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xb:0x0] updating lvb blocks from disk: 2 -> 10 00010000:00010000:0.0:1788845085.769197:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff94a019866000) 00010000:00010000:0.0:1788845085.769200:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:0.0:1788845085.769209:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287aa2f2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6246 expref: 8 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845085.769228:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff94a019866000) 00010000:00010000:0.0:1788845085.769231:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788845085.769235:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6bc00/0xb2187301287aa290 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6215 expref: 7 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845085.769248:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:0.0:1788845085.769250:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788845085.769254:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864600/0xb2187301287aa6db lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a63ff expref: 6 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845085.769266:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845085.769268:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788845085.769272:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6a600/0xb2187301287aa599 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6311 expref: 5 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845085.769288:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff94a019866700) 00010000:00010000:0.0:1788845085.769290:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845085.769292:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864800/0xb2187301287af4f3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa903 expref: 4 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845085.769301:0:5828:0:(ldlm_lock.c:2679:ldlm_export_cancel_locks()) Export ffff94a0090c5800, canceled 5 locks, left on hash table 0. 00010000:00010000:0.0:1788845085.769312:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845085.769314:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845085.769316:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788845085.769693:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1788845086.294217:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845086.930383:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660438144:0@lo:400:kworker.0 00000100:00100000:3.0:1788845086.930459:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845086.932843:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03d5ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660438016:0@lo:400:kworker.0 00000100:00100000:1.0:1788845086.932947:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845086.933008:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03455500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660438272:0@lo:400:kworker.0 00000100:00100000:1.0:1788845086.933039:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845086.933077:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660438016 00000100:00100000:1.0:1788845086.933084:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660438016 00000100:00100000:1.0:1788845086.933127:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:1.0:1788845086.933136:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660438016:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845086.933186:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660438016:12345-0@lo:400:kworker.0 Request processed in 51us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845086.933193:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:1.0:1788845086.933225:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660438144 00000100:00100000:1.0:1788845086.933229:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660438144 00000100:00100000:1.0:1788845086.933255:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660438272 00000100:00100000:1.0:1788845086.933257:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660438272 00000100:00100000:1.0:1788845086.933288:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1788845086.933293:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660438144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845086.934176:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1788845086.934189:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660438272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845086.934246:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660438272:12345-0@lo:400:kworker.0 Request processed in 62us (1206us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845086.934253:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1788845086.934298:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03455500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660438272:0@lo:400:kworker.0 00000100:00100000:0.0:1788845086.936902:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03457480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660438400:0@lo:400:kworker.0 00000100:00100000:0.0:1788845086.936985:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845086.937027:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1788845086.937034:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660438528:0@lo:400:kworker.0 00000100:00100000:0.0:1788845086.937060:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845086.937139:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788845086.937150:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03455500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660438656:0@lo:400:kworker.0 00000100:00100000:2.0:1788845086.937178:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845086.937221:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660438400 00000100:00100000:2.0:1788845086.937229:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660438400 00000100:00100000:2.0:1788845086.937277:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660438656 00000100:00100000:2.0:1788845086.937280:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660438656 00000100:00100000:2.0:1788845086.937303:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1788845086.937313:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660438400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845086.937366:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660438400:12345-0@lo:400:kworker.0 Request processed in 54us (382us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845086.937373:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1788845086.937391:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1788845086.937397:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660438656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845086.937430:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660438656:12345-0@lo:400:kworker.0 Request processed in 33us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845086.937436:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1788845086.937456:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660438528 00000100:00100000:2.0:1788845086.937459:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660438528 00000100:00100000:2.0:1788845086.937481:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1788845086.937486:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660438528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845086.937517:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660438528:12345-0@lo:400:kworker.0 Request processed in 30us (456us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845086.937523:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1788845086.937566:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03455500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660438656:0@lo:400:kworker.0 00000100:00100000:2.0:1788845086.937596:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660438528:0@lo:400:kworker.0 00000100:00100000:2.0:1788845086.937619:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03457480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660438400:0@lo:400:kworker.0 00000100:00100000:1.0:1788845086.937935:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f657100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660438144:12345-0@lo:400:kworker.0 Request processed in 4639us (7475us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845086.937947:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1788845086.937989:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660438016:0@lo:400:kworker.0 00000100:00080000:0.0:1788845086.944200:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788845086.944208:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1788845086.944213:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845086.945734:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03d5dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660438144:0@lo:400:kworker.0 00010000:00010000:0.0:1788845087.314490:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845087.449962:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788845087.449989:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660438784:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788845087.450064:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845087.451601:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660438784 00000100:00100000:2.0:1788845087.451620:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660438784 00000100:00100000:2.0:1788845087.451686:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:2.0:1788845087.451696:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660438784:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845087.451743:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788845087.451947:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660438784:12345-0@lo:13:osp-pre-1-0.0 Request processed in 252us (1882us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845087.451954:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000100:00100000:3.0:1788845087.452421:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660438784:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1788845088.339443:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845088.483992:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788845088.484016:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660438912:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845088.484083:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845088.484147:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660438912 00000100:00100000:0.0:1788845088.484155:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660438912 00000100:00100000:0.0:1788845088.484211:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:0.0:1788845088.484221:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660438912:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845088.484260:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845088.484307:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660438912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845088.484314:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00080000:0.0:1788845088.484335:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845088.484372:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660438912:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845088.539376:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845088.539433:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845088.539461:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845088.539512:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641870848 00000100:00100000:0.0:1788845088.539523:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641870848 00000100:00100000:0.0:1788845088.539588:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 156 00000100:00100000:0.0:1788845088.539599:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641870848:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845088.539683:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641870848:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 86us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845088.539691:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 156 00000100:00080000:0.0:1788845088.539717:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845088.539873:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641870720 00000100:00100000:3.0:1788845088.539878:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641870720 00000100:00100000:3.0:1788845088.539948:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 215 00000100:00100000:3.0:1788845088.539955:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641870720:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845088.539998:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641870720:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 43us (633us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845088.540004:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 215 00000100:00100000:3.0:1788845088.540027:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641870592 00000100:00100000:3.0:1788845088.540030:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641870592 00000100:00100000:3.0:1788845088.540055:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 303 00000100:00100000:3.0:1788845088.540061:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641870592:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845088.540084:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641870592:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 24us (652us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845088.540090:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 303 00000100:00080000:0.0:1788845088.540155:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1788845089.363173:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845090.386196:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1788845090.700141:0:5861:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a 00000001:00080000:2.0:1788845090.700162:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967894 is committed 00000020:00000080:2.0:1788845090.700167:0:5861:0:(genops.c:885:class_export_put()) final put ffff94a0090c5800/5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a 00000020:00000080:3.0:1788845090.700315:0:36:0:(genops.c:844:class_export_destroy()) destroying export ffff94a0090c5800/5aa9ebf6-ce64-4de0-87c4-c2b68ac9584a for lustre-MDT0000 00000020:00000080:3.0:1788845090.700398:0:36:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff94a0090c5000 for lustre-MDT0000 00000001:02000400:1.0:1788845091.116682:0:14692:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg129-client.virtnet: executing check_config_client /mnt/lustre 00010000:00010000:1.0:1788845091.420227:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845092.050136:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660439040:0@lo:400:kworker.0 00000100:00100000:1.0:1788845092.050213:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845092.050277:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660439424:0@lo:400:kworker.0 00000100:00100000:1.0:1788845092.050277:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660439552:0@lo:400:kworker.0 00000100:00100000:2.0:1788845092.050307:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845092.050314:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845092.050347:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660439552 00000100:00100000:2.0:1788845092.050351:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660439424 00000100:00100000:2.0:1788845092.050354:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660439424 00000100:00100000:1.0:1788845092.050354:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660439552 00000100:00100000:2.0:1788845092.050403:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:1.0:1788845092.050403:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1788845092.050412:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660439424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845092.050412:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660439552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845092.050459:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660439424:12345-0@lo:400:kworker.0 Request processed in 47us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845092.050459:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660439552:12345-0@lo:400:kworker.0 Request processed in 48us (142us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845092.050466:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:1.0:1788845092.050466:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:3.0:1788845092.050531:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660439168:0@lo:400:kworker.0 00000100:00100000:0.0:1788845092.050586:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660439040 00000100:00100000:0.0:1788845092.050591:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660439040 00000100:00100000:0.0:1788845092.050620:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1788845092.050626:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660439040:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845092.050670:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660439552:0@lo:400:kworker.0 00000100:00080000:1.0:1788845092.050710:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1788845092.050715:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845092.050739:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660439296:0@lo:400:kworker.0 00000100:00100000:2.0:1788845092.051287:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660439424:0@lo:400:kworker.0 00000100:00100000:3.0:1788845092.052235:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845092.052266:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660439680:0@lo:400:kworker.0 00000100:00100000:2.0:1788845092.052280:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660439168 00000100:00100000:2.0:1788845092.052286:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660439168 00000100:00100000:2.0:1788845092.052326:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1788845092.052410:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660439168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845092.052943:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845092.053011:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660439680 00000100:00100000:3.0:1788845092.053017:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660439680 00000100:00100000:3.0:1788845092.053058:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:3.0:1788845092.053065:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660439680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845092.053109:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660439680:12345-0@lo:400:kworker.0 Request processed in 45us (168us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845092.053116:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:3.0:1788845092.053156:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660439680:0@lo:400:kworker.0 00000100:00080000:3.0:1788845092.053184:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845092.054492:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660439168:12345-0@lo:400:kworker.0 Request processed in 2154us (2255us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845092.054514:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:0.0:1788845092.056091:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660439040:12345-0@lo:400:kworker.0 Request processed in 5458us (5872us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845092.056105:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:3.0:1788845092.056393:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660439168:0@lo:400:kworker.0 00000100:00100000:1.0:1788845092.056700:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845092.056769:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660439040:0@lo:400:kworker.0 00000100:00100000:0.0:1788845092.058492:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660439296 00000100:00100000:0.0:1788845092.058506:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660439296 00000100:00100000:0.0:1788845092.058606:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:0.0:1788845092.058617:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660439296:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845092.058719:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660439296:12345-0@lo:400:kworker.0 Request processed in 104us (2020us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845092.058727:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1788845092.058925:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660439296:0@lo:400:kworker.0 00000100:00080000:3.0:1788845092.059129:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1788845092.434136:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845092.565098:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788845092.565131:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660439808:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845092.565240:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845092.565521:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660439808 00000100:00100000:2.0:1788845092.565533:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660439808 00000100:00100000:2.0:1788845092.565600:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:2.0:1788845092.565612:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660439808:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845092.565661:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788845092.565720:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660439808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 110us (479us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845092.565728:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00100000:2.0:1788845092.565787:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660439808:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1788845093.465983:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845093.587059:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660439936:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845093.587159:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845093.587266:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660439936 00000100:00100000:0.0:1788845093.587275:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660439936 00000100:00100000:0.0:1788845093.587334:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 104 00000100:00100000:0.0:1788845093.587346:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660439936:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845093.587386:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845093.587443:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660439936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 99us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845093.587450:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 104 00000100:00100000:2.0:1788845093.587595:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660439936:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845093.668941:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845093.668978:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845093.669009:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845093.669039:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641871232 00000100:00100000:2.0:1788845093.669049:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641871232 00000100:00100000:2.0:1788845093.669110:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 159 00000100:00100000:2.0:1788845093.669121:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641871232:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845093.669195:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641871232:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 76us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845093.669202:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 159 00000100:00100000:2.0:1788845093.669226:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641871104 00000100:00100000:2.0:1788845093.669229:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641871104 00000100:00100000:2.0:1788845093.669258:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 219 00000100:00100000:2.0:1788845093.669265:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641871104:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845093.669295:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641871104:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 30us (285us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845093.669301:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 219 00000100:00080000:2.0:1788845093.669317:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1788845093.669322:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845093.671380:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641870976 00000100:00100000:3.0:1788845093.671393:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641870976 00000100:00100000:3.0:1788845093.671502:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 305 00000100:00100000:3.0:1788845093.671510:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641870976:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845093.671581:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641870976:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 71us (2650us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845093.671589:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 305 00010000:00010000:0.0:1788845094.483029:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845095.510289:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845096.530768:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845097.170215:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035740000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660440192:0@lo:400:kworker.0 00000100:00100000:3.0:1788845097.170315:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845097.170372:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660440704:0@lo:400:kworker.0 00000100:00100000:3.0:1788845097.170399:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845097.170429:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845097.170435:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035742d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660440320:0@lo:400:kworker.0 00000100:00100000:3.0:1788845097.170458:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845097.170496:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660440192 00000100:00100000:3.0:1788845097.170503:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660440192 00000100:00100000:3.0:1788845097.170548:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660440320 00000100:00100000:3.0:1788845097.170552:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660440320 00000100:00100000:3.0:1788845097.170583:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1788845097.170592:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660440192:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.170861:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660440064:0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.170914:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845097.170938:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a12680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660440576:0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.170962:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845097.170975:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788845097.170980:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660440448:0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.171002:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845097.174942:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660440064 00000100:00100000:1.0:1788845097.174957:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660440064 00000100:00100000:0.0:1788845097.179229:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660440576 00000100:00100000:0.0:1788845097.179249:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660440576 00000100:00100000:0.0:1788845097.179328:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:0.0:1788845097.179339:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660440320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.179421:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660440320:12345-0@lo:400:kworker.0 Request processed in 84us (8961us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845097.179430:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:0.0:1788845097.179450:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:0.0:1788845097.179456:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660440576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.179497:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1875739660440576:12345-0@lo:400:kworker.0 Request processed in 42us (8534us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845097.179504:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:0.0:1788845097.179531:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660440704 00000100:00100000:0.0:1788845097.179536:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660440704 00000100:00100000:0.0:1788845097.179566:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660440448 00000100:00100000:0.0:1788845097.179570:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660440448 00000100:00100000:0.0:1788845097.179590:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1788845097.179597:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660440704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.182086:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660440704:12345-0@lo:400:kworker.0 Request processed in 2480us (11676us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845097.182101:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1788845097.182137:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:0.0:1788845097.182149:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660440448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.182197:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660440448:12345-0@lo:400:kworker.0 Request processed in 53us (11193us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845097.182204:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:3.0:1788845097.182378:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660440192:12345-0@lo:400:kworker.0 Request processed in 11785us (12061us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845097.182388:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00080000:3.0:1788845097.182419:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1788845097.182424:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788845097.182428:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845097.182464:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035742d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660440320:0@lo:400:kworker.0 00000100:00100000:3.0:1788845097.182479:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035740000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660440192:0@lo:400:kworker.0 00000100:00100000:3.0:1788845097.182491:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660440704:0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.183637:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035742a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660440448:0@lo:400:kworker.0 00000100:00100000:0.0:1788845097.183773:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a12680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3314:1875739660440576:0@lo:400:kworker.0 00000100:00100000:1.0:1788845097.185856:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1788845097.185874:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660440064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845097.185985:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3314:x1875739660440064:12345-0@lo:400:kworker.0 Request processed in 117us (15069us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845097.185992:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:0.0:1788845097.187872:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660440064:0@lo:400:kworker.0 00010000:00010000:3.0:1788845097.563017:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845097.688182:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660440832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845097.688252:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845097.688364:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660440832 00000100:00100000:3.0:1788845097.688371:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660440832 00000100:00100000:3.0:1788845097.688418:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:3.0:1788845097.688426:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03411500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660440832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788845097.688462:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788845097.688502:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03411500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660440832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845097.688509:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00000100:00100000:3.0:1788845097.688633:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660440832:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1788845098.578005:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845098.708153:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788845098.708185:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660440960:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845098.708263:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845098.708583:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660440960 00000100:00100000:0.0:1788845098.708592:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660440960 00000100:00100000:0.0:1788845098.708643:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:0.0:1788845098.708652:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660440960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845098.708688:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845098.708746:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660440960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 95us (482us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845098.708753:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:0.0:1788845098.708799:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660440960:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845098.776773:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845098.776821:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845098.776864:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845098.776952:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641871360 00000100:00100000:3.0:1788845098.776962:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641871360 00000100:00100000:3.0:1788845098.777020:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 307 00000100:00100000:3.0:1788845098.777031:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641871360:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845098.777103:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641871360:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 74us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845098.777111:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 307 00000100:00100000:0.0:1788845098.777581:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641871488 00000100:00100000:0.0:1788845098.777587:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641871488 00000100:00080000:3.0:1788845098.777686:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845098.778393:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 223 00000100:00100000:0.0:1788845098.778405:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641871488:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845098.778476:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641871488:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 73us (1713us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845098.778484:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 223 00000100:00100000:0.0:1788845098.781392:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641871616 00000100:00100000:0.0:1788845098.781408:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641871616 00000100:00100000:0.0:1788845098.781656:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 162 00000100:00100000:0.0:1788845098.781667:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641871616:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845098.781752:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641871616:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 87us (4886us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845098.781760:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 162 00000100:00080000:3.0:1788845098.784916:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1788845099.602022:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845100.626638:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845101.651190:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845102.294175:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788845102.294388:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660441216:0@lo:400:kworker.0 00000100:00100000:3.0:1788845102.294465:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845102.294521:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788845102.294539:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660441344:0@lo:400:kworker.0 00000100:00100000:3.0:1788845102.294563:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845102.294599:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660441216 00000100:00100000:3.0:1788845102.294607:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660441216 00000100:00100000:3.0:1788845102.294659:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660441344 00000100:00100000:3.0:1788845102.294662:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660441344 00000100:00100000:3.0:1788845102.294697:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1788845102.294707:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660441216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845102.294761:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660441216:12345-0@lo:400:kworker.0 Request processed in 55us (296us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845102.294769:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1788845102.294786:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:3.0:1788845102.294793:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660441344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845102.294823:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660441344:12345-0@lo:400:kworker.0 Request processed in 30us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845102.294829:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00080000:3.0:1788845102.294849:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845102.294880:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660441344:0@lo:400:kworker.0 00000100:00100000:3.0:1788845102.294922:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660441216:0@lo:400:kworker.0 00000100:00100000:1.0:1788845102.297087:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660441472:0@lo:400:kworker.0 00000100:00100000:1.0:1788845102.297158:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845102.297874:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a0359d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660441088:0@lo:400:kworker.0 00000100:00100000:1.0:1788845102.297930:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845102.297979:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660441088 00000100:00100000:1.0:1788845102.297988:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660441088 00000100:00100000:1.0:1788845102.298031:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:1.0:1788845102.298042:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660441088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845102.298096:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660441088:12345-0@lo:400:kworker.0 Request processed in 57us (165us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845102.298104:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:1.0:1788845102.298152:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a0359d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660441088:0@lo:400:kworker.0 00000100:00100000:3.0:1788845102.298279:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660441472 00000100:00100000:3.0:1788845102.298284:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660441472 00000100:00100000:3.0:1788845102.298326:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1788845102.298346:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660441472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845102.298385:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660441472:12345-0@lo:400:kworker.0 Request processed in 54us (1227us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845102.298391:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00080000:3.0:1788845102.298411:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845102.298614:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660441472:0@lo:400:kworker.0 00000100:00100000:1.0:1788845102.299625:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1788845102.299640:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660441728:0@lo:400:kworker.0 00000100:00100000:1.0:1788845102.299694:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845102.299734:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1788845102.299741:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660441600:0@lo:400:kworker.0 00000100:00100000:1.0:1788845102.299768:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845102.299796:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660441600 00000100:00100000:1.0:1788845102.299801:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660441600 00000100:00100000:1.0:1788845102.299844:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:1.0:1788845102.299851:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660441600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845102.299921:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660441600:12345-0@lo:400:kworker.0 Request processed in 69us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845102.299928:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00080000:1.0:1788845102.299951:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845102.299976:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660441600:0@lo:400:kworker.0 00000100:00100000:2.0:1788845102.300082:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660441728 00000100:00100000:2.0:1788845102.300086:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660441728 00000100:00100000:2.0:1788845102.300132:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:2.0:1788845102.300139:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660441728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845102.300243:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03d5ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660441728:12345-0@lo:400:kworker.0 Request processed in 105us (548us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845102.300250:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00080000:1.0:1788845102.302274:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845102.302324:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660441728:0@lo:400:kworker.0 00010000:00010000:0.0:1788845102.677994:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845102.802379:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788845102.802405:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03412680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660441856:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845102.802488:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845102.802805:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660441856 00000100:00100000:3.0:1788845102.802814:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660441856 00000100:00100000:3.0:1788845102.802876:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:3.0:1788845102.802915:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660441856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788845102.802952:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788845102.803005:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660441856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (516us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845102.803012:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00000100:00100000:1.0:1788845102.803110:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03412680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660441856:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1788845103.698067:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845103.826998:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660441984:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845103.827074:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845103.828311:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660441984 00000100:00100000:3.0:1788845103.828321:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660441984 00000100:00100000:3.0:1788845103.828421:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 108 00000100:00100000:3.0:1788845103.828440:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660441984:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788845103.828476:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788845103.828539:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660441984:12345-0@lo:13:osp-pre-0-0.0 Request processed in 110us (1465us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845103.828546:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 108 00000100:00100000:3.0:1788845103.828592:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0383c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660441984:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788845103.901659:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845103.901708:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845103.901742:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845103.901782:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641871744 00000100:00100000:1.0:1788845103.901791:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641871744 00000100:00100000:1.0:1788845103.901842:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 309 00000100:00100000:1.0:1788845103.901861:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2367:x1875739641871744:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845103.901951:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2367:x1875739641871744:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 92us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845103.901959:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 309 00000100:00100000:1.0:1788845103.901990:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641872000 00000100:00100000:1.0:1788845103.901993:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641872000 00000100:00100000:1.0:1788845103.902030:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 165 00000100:00100000:1.0:1788845103.902037:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641872000:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845103.902066:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641872000:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 31us (324us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845103.902072:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 165 00000100:00080000:1.0:1788845103.902090:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845103.903910:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641871872 00000100:00100000:3.0:1788845103.903925:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641871872 00000100:00100000:3.0:1788845103.904003:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 227 00000100:00100000:3.0:1788845103.904014:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641871872:12345-192.168.201.29@tcp:400:kworker.0 00000100:00080000:1.0:1788845103.904072:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845103.904102:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035be5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2367:x1875739641871872:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 91us (2454us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845103.904111:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 227 00010000:00010000:3.0:1788845104.725075:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845105.746611:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845106.770166:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845107.410137:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660442496:0@lo:400:kworker.0 00000100:00100000:2.0:1788845107.410153:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660442368:0@lo:400:kworker.0 00000100:00100000:1.0:1788845107.410229:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660442240:0@lo:400:kworker.0 00000100:00100000:0.0:1788845107.410302:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845107.410349:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845107.410379:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1788845107.410390:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660442112:0@lo:400:kworker.0 00000100:00100000:2.0:1788845107.410397:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660442496 00000100:00100000:2.0:1788845107.410405:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660442496 00000100:00100000:0.0:1788845107.410423:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845107.410449:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03457480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660442624:0@lo:400:kworker.0 00000100:00100000:0.0:1788845107.410474:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845107.410483:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1788845107.410496:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0383c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660442496:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845107.410520:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660442112 00000100:00100000:0.0:1788845107.410525:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660442112 00000100:00100000:2.0:1788845107.410568:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0383c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660442496:12345-0@lo:400:kworker.0 Request processed in 64us (254us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845107.410575:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:2.0:1788845107.410581:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:0.0:1788845107.410587:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660442112:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1788845107.410608:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788845107.410701:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660442368 00000100:00100000:3.0:1788845107.410706:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660442368 00000100:00100000:3.0:1788845107.410751:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660442624 00000100:00100000:3.0:1788845107.410755:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660442624 00000100:00100000:3.0:1788845107.410776:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1788845107.410783:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03455500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660442368:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845107.410829:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03455500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1875739660442368:12345-0@lo:400:kworker.0 Request processed in 45us (481us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845107.410836:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1788845107.410848:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:3.0:1788845107.410855:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660442624:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845107.410902:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660442624:12345-0@lo:400:kworker.0 Request processed in 26us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845107.410910:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00080000:3.0:1788845107.410934:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845107.410963:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3312:1875739660442368:0@lo:400:kworker.0 00000100:00100000:2.0:1788845107.411279:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660442496:0@lo:400:kworker.0 00000100:00100000:2.0:1788845107.411300:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03457480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660442624:0@lo:400:kworker.0 00000100:00100000:0.0:1788845107.412211:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660442112:12345-0@lo:400:kworker.0 Request processed in 1622us (1783us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845107.412224:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00100000:2.0:1788845107.412348:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660442112:0@lo:400:kworker.0 00000100:00100000:1.0:1788845107.412538:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845107.412639:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03454380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660442752:0@lo:400:kworker.0 00000100:00100000:1.0:1788845107.412668:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845107.414150:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660442752 00000100:00100000:2.0:1788845107.414162:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660442752 00000100:00100000:2.0:1788845107.414257:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1788845107.414267:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660442752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845107.414342:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660442752:12345-0@lo:400:kworker.0 Request processed in 76us (1672us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845107.414350:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:1.0:1788845107.416704:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03454380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660442752:0@lo:400:kworker.0 00000100:00080000:0.0:1788845107.417459:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788845107.421073:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660442240 00000100:00100000:3.0:1788845107.421087:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660442240 00000100:00100000:3.0:1788845107.421159:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:3.0:1788845107.421170:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035689500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660442240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845107.421617:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035689500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660442240:12345-0@lo:400:kworker.0 Request processed in 448us (9078us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845107.421626:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1788845107.430023:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660442240:0@lo:400:kworker.0 00010000:00010000:2.0:1788845107.797431:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845107.923654:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660442880:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788845107.923731:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845107.924342:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660442880 00000100:00100000:3.0:1788845107.924352:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660442880 00000100:00100000:3.0:1788845107.924409:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 109 00000100:00100000:3.0:1788845107.924419:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660442880:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788845107.924469:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788845107.924535:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03457480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660442880:12345-0@lo:13:osp-pre-1-0.0 Request processed in 118us (804us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845107.924543:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 109 00000100:00100000:0.0:1788845107.924650:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660442880:0@lo:13:osp-pre-1-0.0 00000001:02000400:0.0:1788845108.375233:0:15392:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: Using TIMEOUT=20 00010000:00010000:3.0:1788845108.819640:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845108.946173:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660443008:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845108.946250:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845108.946428:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660443008 00000100:00100000:0.0:1788845108.946436:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660443008 00000100:00100000:0.0:1788845108.946483:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 110 00000100:00100000:0.0:1788845108.946498:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03454380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660443008:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845108.946548:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845108.946592:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03454380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660443008:12345-0@lo:13:osp-pre-0-0.0 Request processed in 101us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845108.946599:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 110 00000100:00100000:1.0:1788845108.947677:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660443008:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845109.025371:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845109.025445:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845109.025487:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845109.025499:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641872128 00000100:00100000:0.0:1788845109.025521:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641872128 00000100:00100000:0.0:1788845109.025589:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 311 00000100:00100000:0.0:1788845109.025599:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641872128:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845109.025694:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641872128:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 96us (247us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845109.025702:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 311 00000100:00100000:2.0:1788845109.026143:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641872384 00000100:00100000:2.0:1788845109.026149:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641872384 00000100:00100000:2.0:1788845109.026205:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 168 00000100:00100000:2.0:1788845109.026212:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641872384:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845109.026255:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641872384:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 43us (768us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845109.026262:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 168 00000100:00080000:0.0:1788845109.026930:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845109.031427:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641872256 00000100:00100000:1.0:1788845109.031444:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641872256 00000100:00080000:0.0:1788845109.031632:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845109.032163:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 231 00000100:00100000:1.0:1788845109.032179:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641872256:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845109.032299:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641872256:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 125us (6937us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845109.032307:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 231 00010000:00010000:2.0:1788845109.846621:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845110.866511:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845111.890075:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1788845112.055864:0:15540:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=400866bb len=0 uarg=00007ffe448e07a0 20000000:00000080:2.0:1788845112.056465:0:15540:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=400866bb len=576 karg=ffff949f01193c00 uarg=0000000000000000 20000000:01000000:3.0:1788845112.056578:0:15540:0:(mgs_llog.c:6192:mgs_set_param2()) set_param2 fs='' device='general' param='lod.*.mdt_hash=crush' 20000000:01000000:3.0:1788845112.056585:0:15540:0:(mgs_llog.c:4647:mgs_write_log_param2()) next param 'lod.*.mdt_hash=crush' 20000000:01000000:3.0:1788845112.056590:0:15540:0:(mgs_llog.c:1379:mgs_modify_param()) modify params/general/lod.*.mdt_hash=crush del=0 00000040:00080000:1.0:1788845112.059542:0:15541:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 5 count 7 20000000:00000400:3.0:1788845112.059821:0:15540:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 10000000:01000000:3.0:1788845112.071183:0:15540:0:(mgc_request.c:68:mgc_name2resid()) log params to resid 0x736d61726170/0x3 (params) 00010000:00010000:3.0:1788845112.071217:0:15540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 3/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.071241:0:15540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff94a031d3fe00/0xb2187301287a9ae8 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f98 expref: 9 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.071248:0:15540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff94a008cbba00/0xb2187301287a9af6 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb2187301287a9aef expref: 18 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.071255:0:15540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff94a031e6b200/0xb2187301287a9b04 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb2187301287a9afd expref: 18 pid: 5833 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.071261:0:15540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff94a031e6aa00/0xb2187301287a9b12 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb2187301287a9b0b expref: 18 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.071268:0:15540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 7/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.071311:0:15540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff94a031e6aa00/0xb2187301287a9b12 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb2187301287a9b0b expref: 18 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.071331:0:15540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff94a031e6aa00/0xb2187301287a9b12 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb2187301287a9b0b expref: 18 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845112.071346:0:15540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15540:1875739660443136:0@lo:104: 00000100:00100000:3.0:1788845112.071414:0:15540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1788845112.071525:0:15540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff94a031e6b200/0xb2187301287a9b04 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb2187301287a9afd expref: 18 pid: 5833 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.071536:0:15540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff94a031e6b200/0xb2187301287a9b04 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb2187301287a9afd expref: 18 pid: 5833 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845112.071545:0:15540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15540:1875739660443264:0@lo:104: 00000100:00100000:3.0:1788845112.071566:0:15540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1788845112.074932:0:15540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff94a008cbba00/0xb2187301287a9af6 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb2187301287a9aef expref: 18 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.074951:0:15540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff94a008cbba00/0xb2187301287a9af6 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb2187301287a9aef expref: 18 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845112.074967:0:15540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15540:1875739660443392:0@lo:104: 00000100:00100000:3.0:1788845112.075037:0:15540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845112.075543:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660443136 00000100:00100000:2.0:1788845112.075553:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660443136 00000100:00100000:0.0:1788845112.075637:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660443264 00000100:00100000:0.0:1788845112.075641:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660443264 00000100:00100000:0.0:1788845112.075669:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 25 00000100:00100000:0.0:1788845112.075677:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:05b4ec57-2769-4797-9b3d-21582f14799e+4:15540:x1875739660443136:12345-0@lo:104: 00010000:00010000:0.0:1788845112.075690:0:5823:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.201.129@tcp lock: ffff94a008cbb200/0xb2187301287a9b0b lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xb2187301287a9b12 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.075754:0:5823:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1788845112.075760:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:05b4ec57-2769-4797-9b3d-21582f14799e+4:15540:x1875739660443136:12345-0@lo:104: Request processed in 85us (4346us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845112.075769:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 25 00000100:00100000:0.0:1788845112.075787:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 26 00000100:00100000:0.0:1788845112.075793:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:05b4ec57-2769-4797-9b3d-21582f14799e+3:15540:x1875739660443264:12345-0@lo:104: 00010000:00010000:0.0:1788845112.075800:0:5823:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.201.129@tcp lock: ffff94a031e6ba00/0xb2187301287a9afd lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xb2187301287a9b04 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.075836:0:5823:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:0.0:1788845112.075841:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:05b4ec57-2769-4797-9b3d-21582f14799e+3:15540:x1875739660443264:12345-0@lo:104: Request processed in 49us (4275us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845112.075847:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 26 00010000:00010000:1.0:1788845112.077585:0:5827:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1788845112.077596:0:5827:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.129@tcp lock: ffff94a008cbb200/0xb2187301287a9b0b lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xb2187301287a9b12 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077608:0:5827:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008cbb200 already unused, calling callback (ffffffffc1d3dc20) 10000000:00010000:1.0:1788845112.077611:0:5827:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.129@tcp lock: ffff94a008cbb200/0xb2187301287a9b0b lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0xb2187301287a9b12 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077620:0:5827:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.129@tcp lock: ffff94a008cbb200/0xb2187301287a9b0b lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0xb2187301287a9b12 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1788845112.077626:0:5827:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.129@tcp lock: ffff94a008cbb200/0xb2187301287a9b0b lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0xb2187301287a9b12 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788845112.077632:0:5827:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1788845112.077683:0:5827:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.201.129@tcp lock: ffff94a008cbb200/0xb2187301287a9b0b lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9b12 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077689:0:5827:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1788845112.077712:0:5827:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.129@tcp lock: ffff94a008cbb200/0xb2187301287a9b0b lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9b12 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077722:0:5827:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.129@tcp lock: ffff94a008cbb200/0xb2187301287a9b0b lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9b12 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077739:0:5827:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788845112.077742:0:5827:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.129@tcp lock: ffff94a031e6ba00/0xb2187301287a9afd lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xb2187301287a9b04 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077747:0:5827:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031e6ba00 already unused, calling callback (ffffffffc1d3dc20) 10000000:00010000:1.0:1788845112.077750:0:5827:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.129@tcp lock: ffff94a031e6ba00/0xb2187301287a9afd lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0xb2187301287a9b04 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077761:0:5827:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.129@tcp lock: ffff94a031e6ba00/0xb2187301287a9afd lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0xb2187301287a9b04 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1788845112.077767:0:5827:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.129@tcp lock: ffff94a031e6ba00/0xb2187301287a9afd lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0xb2187301287a9b04 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788845112.077772:0:5827:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1788845112.077786:0:5827:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.201.129@tcp lock: ffff94a031e6ba00/0xb2187301287a9afd lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9b04 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077791:0:5827:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1788845112.077805:0:5827:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.129@tcp lock: ffff94a031e6ba00/0xb2187301287a9afd lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9b04 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077812:0:5827:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.129@tcp lock: ffff94a031e6ba00/0xb2187301287a9afd lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9b04 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.077821:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845112.077822:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845112.077824:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845112.077840:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660443520:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1788845112.077928:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845112.077990:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:05b4ec57-2769-4797-9b3d-21582f14799e:3311:1875739660443648:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1788845112.078021:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845112.081136:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660443520 00000100:00100000:1.0:1788845112.081152:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660443520 00010000:00010000:1.0:1788845112.081190:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff94a031e6aa00/0xb2187301287a9b12 lrc: 5/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb2187301287a9b0b expref: 19 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845112.081247:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660443648 00000100:00100000:1.0:1788845112.081250:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660443648 00010000:00010000:1.0:1788845112.081263:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff94a031e6b200/0xb2187301287a9b04 lrc: 5/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb2187301287a9afd expref: 20 pid: 5833 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845112.081283:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 573 00000100:00100000:1.0:1788845112.081290:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:05b4ec57-2769-4797-9b3d-21582f14799e+20:3314:x1875739660443520:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1788845112.081303:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845112.081308:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff94a031e6aa00/0xb2187301287a9b12 lrc: 5/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb2187301287a9b0b expref: 20 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.081325:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff94a031e6aa00/0xb2187301287a9b12 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb2187301287a9b0b expref: 20 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.081345:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845112.081422:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:05b4ec57-2769-4797-9b3d-21582f14799e+20:3314:x1875739660443520:12345-0@lo:103:ldlm_bl.0 Request processed in 131us (3493us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845112.081429:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 573 00000100:00100000:1.0:1788845112.081444:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 574 00000100:00100000:1.0:1788845112.081449:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3311:x1875739660443648:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1788845112.081458:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845112.081462:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff94a031e6b200/0xb2187301287a9b04 lrc: 5/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb2187301287a9afd expref: 19 pid: 5833 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.081471:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff94a031e6b200/0xb2187301287a9b04 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb2187301287a9afd expref: 19 pid: 5833 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.081493:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845112.081518:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3311:x1875739660443648:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (3497us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845112.081524:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 574 00000100:00100000:1.0:1788845112.081560:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:05b4ec57-2769-4797-9b3d-21582f14799e:3311:1875739660443648:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1788845112.081585:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:05b4ec57-2769-4797-9b3d-21582f14799e:3314:1875739660443520:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1788845112.082527:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845112.083338:0:15540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff94a031d3fe00/0xb2187301287a9ae8 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f98 expref: 9 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845112.083375:0:15540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff94a031d3fe00/0xb2187301287a9ae8 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f98 expref: 9 pid: 5832 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845112.083392:0:15540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15540:1875739660443776:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845112.083462:0:15540:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008964780 going to sleep for 11 seconds 00010000:00010000:3.0:1788845112.083495:0:15540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff94a031e6aa00/0xb2187301287a9b12 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb2187301287a9b0b expref: 18 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845112.083510:0:15540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15540:1875739660443136:0@lo:104: 00010000:00010000:3.0:1788845112.083521:0:15540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff94a031e6b200/0xb2187301287a9b04 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb2187301287a9afd expref: 17 pid: 5833 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845112.083528:0:15540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358d880 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15540:1875739660443264:0@lo:104: 00000100:00100000:0.0:1788845112.084278:0:5823:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1788845112.084295:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660443392 00000100:00100000:0.0:1788845112.084302:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660443392 00000100:00100000:0.0:1788845112.084340:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 27 00000100:00100000:0.0:1788845112.084349:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034553050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:05b4ec57-2769-4797-9b3d-21582f14799e+3:15540:x1875739660443392:12345-0@lo:104: 00010000:00010000:0.0:1788845112.084374:0:5823:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.201.129@tcp lock: ffff94a031d3e600/0xb2187301287a9aef lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0xb2187301287a9af6 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.084444:0:5823:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1788845112.084450:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034553050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:05b4ec57-2769-4797-9b3d-21582f14799e+3:15540:x1875739660443392:12345-0@lo:104: Request processed in 103us (9413us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845112.084457:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 27 00010000:00010000:1.0:1788845112.084624:0:5827:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788845112.084632:0:5827:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.129@tcp lock: ffff94a031d3e600/0xb2187301287a9aef lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0xb2187301287a9af6 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.084640:0:5827:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a031d3e600 already unused, calling callback (ffffffffc1d3dc20) 10000000:00010000:1.0:1788845112.084643:0:5827:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.129@tcp lock: ffff94a031d3e600/0xb2187301287a9aef lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0xb2187301287a9af6 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.084651:0:5827:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.129@tcp lock: ffff94a031d3e600/0xb2187301287a9aef lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0xb2187301287a9af6 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1788845112.084658:0:5827:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.129@tcp lock: ffff94a031d3e600/0xb2187301287a9aef lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0xb2187301287a9af6 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788845112.084664:0:5827:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1788845112.084701:0:5827:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.201.129@tcp lock: ffff94a031d3e600/0xb2187301287a9aef lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9af6 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.084708:0:5827:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1788845112.084727:0:5827:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.129@tcp lock: ffff94a031d3e600/0xb2187301287a9aef lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9af6 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.084733:0:5827:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.129@tcp lock: ffff94a031d3e600/0xb2187301287a9aef lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb2187301287a9af6 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.084752:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845112.084754:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845112.084755:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845112.084768:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:05b4ec57-2769-4797-9b3d-21582f14799e:3312:1875739660443904:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1788845112.084802:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845112.084847:0:5825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660443904 00000100:00100000:1.0:1788845112.084851:0:5825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660443904 00010000:00010000:1.0:1788845112.084870:0:5825:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff94a008cbba00/0xb2187301287a9af6 lrc: 5/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb2187301287a9aef expref: 17 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845112.084912:0:5825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 575 00000100:00100000:1.0:1788845112.084919:0:5825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:05b4ec57-2769-4797-9b3d-21582f14799e+17:3312:x1875739660443904:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1788845112.084936:0:5825:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845112.084942:0:5825:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff94a008cbba00/0xb2187301287a9af6 lrc: 5/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb2187301287a9aef expref: 17 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.084956:0:5825:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff94a008cbba00/0xb2187301287a9af6 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb2187301287a9aef expref: 17 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845112.084970:0:5825:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845112.085006:0:5825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:05b4ec57-2769-4797-9b3d-21582f14799e+17:3312:x1875739660443904:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (196us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845112.085022:0:5825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 575 00000100:00100000:1.0:1788845112.085056:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02480380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:05b4ec57-2769-4797-9b3d-21582f14799e:3312:1875739660443904:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1788845112.085090:0:15540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff94a008cbba00/0xb2187301287a9af6 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb2187301287a9aef expref: 16 pid: 5832 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845112.085106:0:15540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15540:1875739660443392:0@lo:104: 00000100:00100000:3.0:1788845112.086492:0:15540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15540:1875739660443776:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845112.086525:0:15540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 3/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845112.087393:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845112.087502:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641872512 00000100:00100000:0.0:1788845112.087510:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641872512 00010000:00010000:0.0:1788845112.087543:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff94a031d3fe00/0xb2187301287a9ae8 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f98 expref: 10 pid: 5832 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845112.087582:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 576 00000100:00100000:0.0:1788845112.087591:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0082fd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2370:x1875739641872512:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788845112.087604:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788845112.087609:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff94a031d3fe00/0xb2187301287a9ae8 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f98 expref: 10 pid: 5832 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087626:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff94a031d3fe00/0xb2187301287a9ae8 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f98 expref: 10 pid: 5832 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087642:0:13304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff94a031d3fe00/0xb2187301287a9ae8 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109a5f98 expref: 10 pid: 5832 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087657:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 3/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087665:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 4/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087679:0:13304:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 5/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087686:0:13304:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 5/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087693:0:13304:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 4/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087701:0:13304:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 5/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087711:0:13304:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216fa00 already unused, calling callback (ffffffffc1717350) 00010000:00010000:0.0:1788845112.087715:0:13304:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 5/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087723:0:13304:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 6/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087743:0:13304:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 4/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845112.087753:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788845112.087807:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0082fd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+9:2370:x1875739641872512:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 216us (416us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845112.087815:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 576 00010000:00010000:2.0:1788845112.089245:0:15540:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 1/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845112.089258:0:15540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 1/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845112.089264:0:15540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff94a04216fa00/0xb2187301287af516 lrc: 0/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 15540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845112.530939:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3313:1875739660444032:0@lo:400:kworker.0 00000100:00100000:0.0:1788845112.530973:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660444288:0@lo:400:kworker.0 00000100:00100000:1.0:1788845112.531018:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845112.531087:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845112.531094:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660444544:0@lo:400:kworker.0 00000100:00100000:0.0:1788845112.531125:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788845112.531129:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845112.531135:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3311:1875739660444416:0@lo:400:kworker.0 00000100:00100000:1.0:1788845112.531159:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1788845112.531166:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845112.531170:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660444160:0@lo:400:kworker.0 00000100:00100000:1.0:1788845112.531202:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845112.531229:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660444288 00000100:00100000:0.0:1788845112.531238:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660444288 00000100:00100000:1.0:1788845112.531242:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660444032 00000100:00100000:1.0:1788845112.531249:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660444032 00000100:00100000:0.0:1788845112.531294:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660444544 00000100:00100000:0.0:1788845112.531298:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660444544 00000100:00100000:0.0:1788845112.531320:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:0.0:1788845112.531329:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660444288:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845112.531362:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660444416 00000100:00100000:1.0:1788845112.531367:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660444416 00000100:00100000:1.0:1788845112.531386:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660444160 00000100:00100000:1.0:1788845112.531389:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660444160 00000100:00100000:1.0:1788845112.531405:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:1.0:1788845112.531411:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3313:x1875739660444032:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845112.531457:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3313:x1875739660444032:12345-0@lo:400:kworker.0 Request processed in 45us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845112.531464:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:1.0:1788845112.531498:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:1.0:1788845112.531507:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1875739660444416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845112.531521:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660444288:12345-0@lo:400:kworker.0 Request processed in 193us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845112.531534:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1788845112.531549:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1875739660444416:12345-0@lo:400:kworker.0 Request processed in 43us (382us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845112.531555:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:1.0:1788845112.531563:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:0.0:1788845112.531567:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660444544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845112.531584:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:1.0:1788845112.531594:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660444160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845112.531604:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02482d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660444544:12345-0@lo:400:kworker.0 Request processed in 40us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845112.531611:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00080000:0.0:1788845112.531632:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788845112.531635:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660444160:12345-0@lo:400:kworker.0 Request processed in 42us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845112.531643:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:1.0:1788845112.531686:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660444160:0@lo:400:kworker.0 00000100:00100000:1.0:1788845112.531698:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3313:1875739660444032:0@lo:400:kworker.0 00000100:00080000:0.0:1788845112.531706:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788845112.531710:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660444544:0@lo:400:kworker.0 00000100:00080000:0.0:1788845112.531710:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845112.531740:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3311:1875739660444416:0@lo:400:kworker.0 00000100:00100000:0.0:1788845112.531756:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660444288:0@lo:400:kworker.0 00010000:00010000:1.0:1788845112.915269:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845113.044225:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660444672:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788845113.044297:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.044388:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660444672 00000100:00100000:2.0:1788845113.044395:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660444672 00000100:00100000:2.0:1788845113.044441:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:2.0:1788845113.044450:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660444672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845113.044527:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788845113.045069:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660444672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 619us (770us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845113.045079:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:1.0:1788845113.045247:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660444672:0@lo:13:osp-pre-1-0.0 10000000:01000000:2.0:1788845113.876693:0:5836:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1788845113.876702:0:5836:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff94a03d5bc000 from 7 10000000:01000000:2.0:1788845113.876707:0:5836:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1788845113.876757:0:5836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.129@tcp lock: ffff94a008864800/0xb2187301287af51d lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.876770:0:5836:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.129@tcp lock: ffff94a008864800/0xb2187301287af51d lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.876782:0:5836:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.129@tcp lock: ffff94a008864800/0xb2187301287af51d lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845113.876809:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9a680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660444800:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.876880:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.876965:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009273780 going to sleep for 11 seconds 00000100:00100000:0.0:1788845113.877049:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660444800 00000100:00100000:0.0:1788845113.877057:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660444800 00000100:00100000:0.0:1788845113.877115:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:0.0:1788845113.877124:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+16:5836:x1875739660444800:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1788845113.877138:0:5832:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845113.877152:0:5832:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff949f02006400/0xb2187301287af524 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0xb2187301287af51d expref: -99 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845113.877175:0:5832:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff949f02006400/0xb2187301287af524 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb2187301287af51d expref: 17 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845113.877185:0:5832:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff949f02006400/0xb2187301287af524 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb2187301287af51d expref: 17 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845113.877194:0:5832:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006400, rc 0) 00000100:00100000:0.0:1788845113.877246:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+17:5836:x1875739660444800:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 123us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845113.877252:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1788845113.877697:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9a680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660444800:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1788845113.877712:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.129@tcp lock: ffff94a008864800/0xb2187301287af51d lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af524 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845113.877722:0:5836:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.129@tcp lock: ffff94a008864800/0xb2187301287af51d lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xb2187301287af524 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845113.877781:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660444928:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1788845113.877810:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845113.877831:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a00314c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845113.878546:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660444928 00000100:00100000:2.0:1788845113.878564:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660444928 00000100:00100000:2.0:1788845113.878607:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:2.0:1788845113.878615:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+17:5836:x1875739660444928:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.878857:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+17:5836:x1875739660444928:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 242us (1045us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845113.878865:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:2.0:1788845113.878926:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358fb80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660444928:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.878961:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445056:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.878995:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.879015:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0092736c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845113.879026:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660445056 00000100:00100000:2.0:1788845113.879029:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660445056 00000100:00100000:2.0:1788845113.879045:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1788845113.879050:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+17:5836:x1875739660445056:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.879174:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+17:5836:x1875739660445056:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 123us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845113.879182:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1788845113.879218:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445056:0@lo:503:ll_cfg_requeue.0 00000040:00080000:2.0:1788845113.879814:0:15598:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:2.0:1788845113.879867:0:5836:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:2.0:1788845113.879919:0:5836:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.201.129@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:2.0:1788845113.879928:0:5836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.129@tcp lock: ffff94a008864800/0xb2187301287af51d lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af524 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.879937:0:5836:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.129@tcp lock: ffff94a008864800/0xb2187301287af51d lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af524 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788845113.879947:0:5836:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1788845113.879949:0:5836:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff94a010585800 from 7 10000000:01000000:2.0:1788845113.879952:0:5836:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1788845113.879976:0:5836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.129@tcp lock: ffff94a008864600/0xb2187301287af52b lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.879983:0:5836:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.129@tcp lock: ffff94a008864600/0xb2187301287af52b lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.879991:0:5836:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.129@tcp lock: ffff94a008864600/0xb2187301287af52b lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845113.880004:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445184:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.880037:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.880058:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009273cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845113.880070:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660445184 00000100:00100000:2.0:1788845113.880074:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660445184 00000100:00100000:2.0:1788845113.880094:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:2.0:1788845113.880100:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00912c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+17:5836:x1875739660445184:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1788845113.880111:0:5834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845113.880122:0:5834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff94a008865400/0xb2187301287af532 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0xb2187301287af52b expref: -99 pid: 5834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.880142:0:5834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff94a008865400/0xb2187301287af532 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb2187301287af52b expref: 18 pid: 5834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.880152:0:5834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff94a008865400/0xb2187301287af532 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb2187301287af52b expref: 18 pid: 5834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.880160:0:5834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865400, rc 0) 00000100:00100000:2.0:1788845113.880189:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00912c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+18:5836:x1875739660445184:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 90us (152us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845113.880196:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00100000:2.0:1788845113.880227:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445184:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1788845113.880237:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.129@tcp lock: ffff94a008864600/0xb2187301287af52b lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af532 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.880245:0:5836:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.129@tcp lock: ffff94a008864600/0xb2187301287af52b lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xb2187301287af532 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845113.880285:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445312:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.880311:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.880323:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009273cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845113.880333:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660445312 00000100:00100000:2.0:1788845113.880336:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660445312 00000100:00100000:2.0:1788845113.880351:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:2.0:1788845113.880357:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00911f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+18:5836:x1875739660445312:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.880547:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00911f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+18:5836:x1875739660445312:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 189us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845113.880553:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:2.0:1788845113.880581:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445312:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.880604:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445440:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.880640:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.880654:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009273cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788845113.880664:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660445440 00000100:00100000:2.0:1788845113.880667:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660445440 00000100:00100000:2.0:1788845113.880683:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:2.0:1788845113.880688:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00911d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+18:5836:x1875739660445440:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.880807:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00911d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+18:5836:x1875739660445440:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 118us (166us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845113.880813:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:2.0:1788845113.880848:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445440:0@lo:503:ll_cfg_requeue.0 00000040:00080000:0.0:1788845113.893385:0:15599:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:2.0:1788845113.896578:0:5836:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:2.0:1788845113.896613:0:5836:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.201.129@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:2.0:1788845113.896626:0:5836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.129@tcp lock: ffff94a008864600/0xb2187301287af52b lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af532 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.896641:0:5836:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.129@tcp lock: ffff94a008864600/0xb2187301287af52b lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af532 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788845113.896652:0:5836:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1788845113.896655:0:5836:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff94a008856800 from 7 10000000:01000000:2.0:1788845113.896658:0:5836:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1788845113.896696:0:5836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.129@tcp lock: ffff94a008865a00/0xb2187301287af539 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.896704:0:5836:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.129@tcp lock: ffff94a008865a00/0xb2187301287af539 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.896714:0:5836:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.129@tcp lock: ffff94a008865a00/0xb2187301287af539 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845113.896739:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445568:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.896803:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.896837:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009273cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845113.897034:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660445568 00000100:00100000:3.0:1788845113.897043:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660445568 00000100:00100000:3.0:1788845113.897093:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:3.0:1788845113.897103:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00911c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+18:5836:x1875739660445568:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1788845113.897117:0:5834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845113.897129:0:5834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff94a04216e400/0xb2187301287af540 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0xb2187301287af539 expref: -99 pid: 5834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845113.897151:0:5834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff94a04216e400/0xb2187301287af540 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb2187301287af539 expref: 19 pid: 5834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845113.897163:0:5834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff94a04216e400/0xb2187301287af540 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb2187301287af539 expref: 19 pid: 5834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845113.897172:0:5834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e400, rc 0) 00000100:00100000:3.0:1788845113.897215:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00911c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:5836:x1875739660445568:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 113us (412us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845113.897222:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:2.0:1788845113.897332:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445568:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1788845113.897347:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.129@tcp lock: ffff94a008865a00/0xb2187301287af539 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af540 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845113.897356:0:5836:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.129@tcp lock: ffff94a008865a00/0xb2187301287af539 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0xb2187301287af540 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845113.897406:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445696:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.897433:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.897460:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009273cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788845113.897576:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660445696 00000100:00100000:3.0:1788845113.897580:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660445696 00000100:00100000:3.0:1788845113.897608:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 319 00000100:00100000:3.0:1788845113.897614:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00911f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:5836:x1875739660445696:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1788845113.897854:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00911f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:5836:x1875739660445696:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 240us (420us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845113.897861:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 319 00000100:00100000:2.0:1788845113.898071:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445696:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.898100:0:5836:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445824:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1788845113.898139:0:5836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845113.898157:0:5836:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a009273cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845113.898413:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660445824 00000100:00100000:0.0:1788845113.898418:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660445824 00000100:00100000:0.0:1788845113.898462:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 320 00000100:00100000:0.0:1788845113.898469:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:5836:x1875739660445824:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1788845113.899417:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:5836:x1875739660445824:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 944us (1273us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845113.899427:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 320 00000100:00100000:2.0:1788845113.903687:0:5836:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:05b4ec57-2769-4797-9b3d-21582f14799e:5836:1875739660445824:0@lo:503:ll_cfg_requeue.0 00000040:00080000:2.0:1788845113.904911:0:15602:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:3.0:1788845113.905243:0:5836:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:3.0:1788845113.905262:0:5836:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.201.129@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:3.0:1788845113.905273:0:5836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.129@tcp lock: ffff94a008865a00/0xb2187301287af539 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af540 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845113.905285:0:5836:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.129@tcp lock: ffff94a008865a00/0xb2187301287af539 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb2187301287af540 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845113.945801:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845113.971841:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845113.971901:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641872640 00000100:00100000:1.0:1788845113.971911:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641872640 00000100:00100000:1.0:1788845113.971963:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 321 00000100:00100000:1.0:1788845113.971973:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+9:5608:x1875739641872640:12345-192.168.201.29@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788845113.971991:0:5832:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845113.972009:0:5832:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff949f036c0c00/0xb2187301287af547 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0x3100577d109aa926 expref: -99 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845113.972037:0:5832:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff949f036c0c00/0xb2187301287af547 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa926 expref: 10 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845113.972048:0:5832:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff949f036c0c00/0xb2187301287af547 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa926 expref: 10 pid: 5832 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845113.972057:0:5832:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0c00, rc 0) 00000100:00100000:1.0:1788845113.972109:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:5608:x1875739641872640:12345-192.168.201.29@tcp:101:ll_cfg_requeue.0 Request processed in 137us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845113.972116:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 321 00000100:00100000:1.0:1788845113.977775:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845113.977869:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641872768 00000100:00100000:0.0:1788845113.977878:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641872768 00000100:00100000:0.0:1788845113.977950:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 322 00000100:00100000:0.0:1788845113.977960:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:5608:x1875739641872768:12345-192.168.201.29@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1788845113.978209:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:5608:x1875739641872768:12345-192.168.201.29@tcp:501:ll_cfg_requeue.0 Request processed in 249us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845113.978217:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 322 00000100:00100000:1.0:1788845113.983949:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845113.984069:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641872896 00000100:00100000:3.0:1788845113.984079:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641872896 00000100:00100000:3.0:1788845113.984140:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 323 00000100:00100000:3.0:1788845113.984150:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:5608:x1875739641872896:12345-192.168.201.29@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1788845113.984354:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:5608:x1875739641872896:12345-192.168.201.29@tcp:503:ll_cfg_requeue.0 Request processed in 205us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845113.984362:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 323 00000100:00100000:0.0:1788845114.068910:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660445952:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845114.068977:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845114.069043:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660445952 00000100:00100000:0.0:1788845114.069050:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660445952 00000100:00100000:0.0:1788845114.069099:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 112 00000100:00100000:0.0:1788845114.069107:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660445952:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845114.069142:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845114.069182:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660445952:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (204us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845114.069193:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 112 00000100:00080000:0.0:1788845114.069214:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845114.069248:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660445952:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845114.143657:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845114.143699:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845114.143727:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641873024 00000100:00100000:0.0:1788845114.143736:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641873024 00000100:00100000:0.0:1788845114.143800:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 235 00000100:00100000:0.0:1788845114.143811:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641873024:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845114.143924:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641873024:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 115us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845114.143931:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 235 00000100:00100000:0.0:1788845114.143956:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641873152 00000100:00100000:0.0:1788845114.143960:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641873152 00000100:00100000:0.0:1788845114.143989:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 171 00000100:00100000:0.0:1788845114.143996:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641873152:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845114.144024:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641873152:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 29us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845114.144030:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 171 00000100:00080000:0.0:1788845114.144046:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788845114.144051:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788845114.963224:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845115.986481:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788845116.334515:0:15737:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg129-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 00010000:00010000:1.0:1788845117.010345:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845117.653515:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f98380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660446464:0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.653588:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845117.653664:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788845117.653670:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:05b4ec57-2769-4797-9b3d-21582f14799e:3311:1875739660446080:0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.653700:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845117.653721:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f99500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3311:1875739660446592:0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.653745:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845117.653917:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3313:1875739660446208:0@lo:400:kworker.0 00000100:00100000:3.0:1788845117.653947:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845117.653977:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660446720:0@lo:400:kworker.0 00000100:00100000:3.0:1788845117.653999:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845117.654017:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788845117.654023:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01f9bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660446336:0@lo:400:kworker.0 00000100:00100000:3.0:1788845117.654042:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845117.654073:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660446592 00000100:00100000:3.0:1788845117.654081:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660446592 00000100:00100000:3.0:1788845117.654131:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660446208 00000100:00100000:3.0:1788845117.654134:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660446208 00000100:00100000:3.0:1788845117.654147:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660446336 00000100:00100000:3.0:1788845117.654149:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660446336 00000100:00100000:3.0:1788845117.654166:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:3.0:1788845117.654174:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1875739660446592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.654408:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1788845117.654417:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3313:x1875739660446208:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.654465:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3313:x1875739660446208:12345-0@lo:400:kworker.0 Request processed in 50us (517us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845117.654473:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1788845117.654491:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:0.0:1788845117.654497:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660446336:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.654525:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660446336:12345-0@lo:400:kworker.0 Request processed in 28us (481us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845117.654531:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00080000:0.0:1788845117.654581:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845117.654595:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660446464 00000100:00100000:0.0:1788845117.654599:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660446464 00000100:00100000:0.0:1788845117.654627:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660446720 00000100:00100000:0.0:1788845117.654630:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660446720 00000100:00100000:0.0:1788845117.654649:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:0.0:1788845117.654655:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660446464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.654691:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660446464:12345-0@lo:400:kworker.0 Request processed in 36us (1102us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845117.654697:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:0.0:1788845117.654711:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:0.0:1788845117.654716:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660446720:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.654744:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660446720:12345-0@lo:400:kworker.0 Request processed in 28us (744us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845117.654751:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00080000:0.0:1788845117.654770:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788845117.654773:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845117.654799:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f98380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660446464:0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.654844:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660446336:0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.654852:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3313:1875739660446208:0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.654859:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660446720:0@lo:400:kworker.0 00000100:00100000:3.0:1788845117.655139:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1875739660446592:12345-0@lo:400:kworker.0 Request processed in 965us (1392us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845117.655147:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:1.0:1788845117.656473:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660446080 00000100:00100000:1.0:1788845117.656485:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660446080 00000100:00100000:1.0:1788845117.656545:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1788845117.656565:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3311:x1875739660446080:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845117.656637:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3311:x1875739660446080:12345-0@lo:400:kworker.0 Request processed in 74us (2935us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845117.656645:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:0.0:1788845117.656767:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f9b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:05b4ec57-2769-4797-9b3d-21582f14799e:3311:1875739660446080:0@lo:400:kworker.0 00000100:00100000:0.0:1788845117.656782:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01f99500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3311:1875739660446592:0@lo:400:kworker.0 00010000:00010000:0.0:1788845118.034173:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845118.176587:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788845118.176611:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660446848:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788845118.176679:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845118.176750:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660446848 00000100:00100000:3.0:1788845118.176758:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660446848 00000100:00100000:3.0:1788845118.176806:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:3.0:1788845118.176815:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660446848:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788845118.176852:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788845118.176924:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660446848:12345-0@lo:13:osp-pre-1-0.0 Request processed in 110us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845118.176931:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00100000:3.0:1788845118.176980:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f0358c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660446848:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1788845119.058463:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845119.186150:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660446976:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788845119.186244:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845119.187142:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660446976 00000100:00100000:0.0:1788845119.187151:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660446976 00000100:00100000:0.0:1788845119.187272:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 114 00000100:00100000:0.0:1788845119.187283:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660446976:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845119.187325:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788845119.187579:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02481180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660446976:12345-0@lo:13:osp-pre-0-0.0 Request processed in 298us (1334us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845119.187587:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 114 00000100:00100000:0.0:1788845119.187639:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660446976:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1788845119.187706:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845119.261300:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845119.261365:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845119.261451:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845119.261551:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641873536 00000100:00100000:0.0:1788845119.261560:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641873536 00000100:00100000:0.0:1788845119.261618:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 174 00000100:00100000:0.0:1788845119.261627:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641873536:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845119.261696:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02480700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641873536:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 70us (341us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845119.261702:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 174 00000100:00080000:1.0:1788845119.261873:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845119.262315:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641873280 00000100:00100000:2.0:1788845119.262322:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641873280 00000100:00100000:2.0:1788845119.262373:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 325 00000100:00100000:2.0:1788845119.262381:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641873280:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845119.262430:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641873280:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 50us (1142us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845119.262437:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 325 00000100:00100000:3.0:1788845119.268535:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641873408 00000100:00100000:3.0:1788845119.268552:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641873408 00000100:00100000:2.0:1788845119.269263:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 239 00000100:00100000:2.0:1788845119.269275:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641873408:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845119.269383:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2368:x1875739641873408:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 110us (7930us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845119.269391:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 239 00000100:00080000:1.0:1788845119.269558:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1788845120.083092:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788845120.905377:0:15837:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: === sanity: finish setup 01:25:19 (1788845119) === 00010000:00010000:1.0:1788845121.106644:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845121.146200:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845121.146274:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641873664 00000100:00100000:0.0:1788845121.146284:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641873664 00000100:00100000:0.0:1788845121.146378:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1692 00000100:00100000:0.0:1788845121.146389:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:15461:x1875739641873664:12345-192.168.201.29@tcp:101:bash.0 00010000:00010000:0.0:1788845121.146410:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845121.146472:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af54e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa92d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.146541:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.146570:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.146583:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.146618:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.146626:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.146632:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.146849:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1788845121.147310:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1788845121.147370:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147378:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147384:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.147388:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147396:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147408:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:0.0:1788845121.147411:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.147415:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147420:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af555 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147432:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147438:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147444:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147448:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.147451:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147456:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147463:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x54ef8a6e (ffff94a01929ca00) 00010000:00010000:0.0:1788845121.147465:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.147468:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147473:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af55c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147509:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af54e lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa92d expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147519:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:0.0:1788845121.147521:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788845121.147524:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af54e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa92d expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.147533:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007600, rc 0) 00000100:00100000:0.0:1788845121.147702:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:15461:x1875739641873664:12345-192.168.201.29@tcp:101:bash.0 Request processed in 1306us (1504us total) trans 0 rc 301/301 00000100:00100000:0.0:1788845121.147710:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1692 00000100:00100000:3.0:1788845121.159158:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.160173:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641873792 00000100:00100000:2.0:1788845121.160189:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641873792 00000100:00100000:2.0:1788845121.160270:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1693 00000100:00100000:2.0:1788845121.160282:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:15461:x1875739641873792:12345-192.168.201.29@tcp:101:bash.0 00010000:00010000:2.0:1788845121.160306:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.160345:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af563 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa934 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160413:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160427:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160439:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160493:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160502:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160509:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160718:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160727:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160734:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160741:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.160745:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160753:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160764:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:2.0:1788845121.160767:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.160771:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160777:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af56a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160789:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160796:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160802:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160807:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.160810:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160815:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160826:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x77bdd2c7 (ffff94a0087c6400) 00010000:00010000:2.0:1788845121.160829:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.160832:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160837:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af571 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160866:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af563 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa934 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160876:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:2.0:1788845121.160878:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788845121.160908:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af563 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa934 expref: 16 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.160919:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864e00, rc 0) 00000100:00100000:2.0:1788845121.160988:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0359d4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:15461:x1875739641873792:12345-192.168.201.29@tcp:101:bash.0 Request processed in 708us (1845us total) trans 0 rc 301/301 00000100:00100000:2.0:1788845121.160995:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1693 00000100:00100000:1.0:1788845121.167948:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.168090:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641873920 00000100:00100000:2.0:1788845121.168101:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641873920 00000100:00100000:2.0:1788845121.168162:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 44 00000100:00100000:2.0:1788845121.168174:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:15461:x1875739641873920:12345-192.168.201.29@tcp:37:bash.0 00000100:00100000:1.0:1788845121.175150:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:15461:x1875739641873920:12345-192.168.201.29@tcp:37:bash.0 Request processed in 6965us (7198us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845121.175165:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 44 00000100:00100000:0.0:1788845121.186747:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.187039:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641874048 00000100:00100000:2.0:1788845121.187058:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641874048 00000100:00100000:2.0:1788845121.187137:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1694 00000100:00100000:2.0:1788845121.187152:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:18208:x1875739641874048:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:2.0:1788845121.187181:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.187220:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af578 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa93b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.187310:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.187340:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.187356:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.187373:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.188375:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.189006:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.196771:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af58d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.196793:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af58d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.196807:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af58d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.196852:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x5:0x0]: returning size 5477732 00000004:00200000:3.0:1788845121.196857:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.196880:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af58d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845121.196932:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:3.0:1788845121.196936:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb mtime from disk: 0 -> 1788844946 00000004:00010000:3.0:1788845121.196939:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb atime from disk: 0 -> 1788844945 00000004:00010000:3.0:1788845121.196942:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb ctime from disk: 0 -> 1788844946 00000004:00010000:3.0:1788845121.196944:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb blocks from disk: 18427282363563114435 -> 2 00002000:00010000:3.0:1788845121.196948:0:5839:0:(mdt_io.c:1550:mdt_lvb2reply()) size 0 00010000:00010000:3.0:1788845121.196955:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.196963:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.196971:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.196978:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.196984:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.196993:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197008:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:3.0:1788845121.197011:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.197016:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197024:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af57f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197044:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197051:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197057:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197062:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.197065:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197071:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197080:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x98980ae (ffff94a0087c6400) 00010000:00010000:3.0:1788845121.197082:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.197087:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197093:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af586 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197134:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af578 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa93b expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197153:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af58d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa93b expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.197167:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff94a00be8de00) 00010000:00010000:3.0:1788845121.197171:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788845121.197174:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:3.0:1788845121.197251:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:18208:x1875739641874048:12345-192.168.201.29@tcp:101:rm.0 Request processed in 10101us (10512us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.197263:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1694 00000100:00100000:1.0:1788845121.202976:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845121.203021:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641874176 00000100:00100000:1.0:1788845121.203034:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641874176 00000100:00100000:1.0:1788845121.203087:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1279 00000100:00100000:1.0:1788845121.203098:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0+-99:2366:x1875739641874176:12345-192.168.201.29@tcp:8:rm.0 00010000:00080000:1.0:1788845121.203135:0:6682:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from 69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp t0 exp 0000000000000000 cur 510 last 0 00000020:00000080:1.0:1788845121.203177:0:6682:0:(genops.c:1385:class_connect()) connect: client 69859fee-4322-4bed-b50b-7a298318b0f6, cookie 0xb2187301287af594 00002000:00100000:1.0:1788845121.203192:0:6682:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli 69859fee-4322-4bed-b50b-7a298318b0f6/ffff949f02763000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396c ocd_grant: 8519680 ocd_index: 0 ocd_group 0 00002000:00100000:1.0:1788845121.203222:0:6682:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:1.0:1788845121.203546:0:6682:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:1.0:1788845121.203591:0:6682:0:(import.c:65:import_set_state_nolock()) ffff949f02760000 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1788845121.203684:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2366:x1875739641874176:12345-192.168.201.29@tcp:8:rm.0 Request processed in 585us (721us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845121.203694:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1279 00000100:00100000:1.0:1788845121.220048:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845121.220097:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641874304 00000100:00100000:1.0:1788845121.220106:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641874304 00000100:00100000:1.0:1788845121.220164:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1280 00000100:00100000:1.0:1788845121.220175:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:0:x1875739641874304:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:1.0:1788845121.220194:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845121.220216:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa942 expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788845121.220248:0:7983:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x2:0x0] initial LVB size: 5477732, mtime: 0x6a9f9b92, atime: 0x6a9f9b92, ctime: 0x6a9f9b92, blocks: 0x1007 00010000:00010000:1.0:1788845121.220266:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2:0x0].0x0 since=510s v=(0 0) 00010000:00010000:1.0:1788845121.220271:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788845121.220283:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.220303:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.220333:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0600, rc 0) 00000100:00100000:1.0:1788845121.220393:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:0:x1875739641874304:12345-192.168.201.29@tcp:101:rm.0 Request processed in 220us (355us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845121.220400:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1280 00000100:00080000:1.0:1788845121.220429:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845121.229650:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.229881:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641874560 00000100:00100000:3.0:1788845121.229998:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641874560 00000100:00100000:3.0:1788845121.230048:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1695 00000100:00100000:3.0:1788845121.230058:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:18208:x1875739641874560:12345-192.168.201.29@tcp:36:rm.0 00010000:00010000:3.0:1788845121.230168:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788845121.230221:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af58d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa93b expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.230240:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff94a00be8de00) 00010000:00010000:3.0:1788845121.230421:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1400/0xb2187301287aaf0f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6b61 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.230433:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6600) 00010000:00010000:3.0:1788845121.230440:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845121.230460:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.230473:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.230483:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.230497:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.230504:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.230510:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.234040:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.234064:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.234186:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788845121.240448:0:5840:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].2, 64 off8320 00000004:00080000:3.0:1788845121.240466:0:5840:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 510437582146ns, next at 511437578410ns, rc = 0 00010000:00010000:3.0:1788845121.241139:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241158:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241170:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241180:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241193:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241380:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241403:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788845121.241430:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241436:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241444:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241450:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216e800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.241455:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241463:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241476:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff94a00be8d200) 00010000:00010000:3.0:1788845121.241480:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788845121.241484:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241491:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af5b0 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241511:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241518:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241525:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241531:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216ee00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.241533:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241539:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241975:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:3.0:1788845121.241981:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.241986:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.241995:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5a2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845121.242014:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a5b515ce reply state 000000009c4c4a7a transno 4294967895 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.242024:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845121.242618:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:18208:x1875739641874560:12345-192.168.201.29@tcp:36:rm.0 Request processed in 12556us (12973us total) trans 4294967895 rc 0/0 00000100:00100000:3.0:1788845121.242628:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1695 00010000:00010000:3.0:1788845121.242701:0:5827:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788845121.242707:0:5827:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.242716:0:5827:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a0206c4200 already unused, calling callback (ffffffffc1717350) 00010000:00010000:3.0:1788845121.242720:0:5827:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.242727:0:5827:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.242738:0:5827:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff94a00be8d200) 00010000:00010000:3.0:1788845121.242740:0:5827:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845121.242744:0:5827:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.242750:0:5827:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5b7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.242762:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845121.242764:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845121.242766:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845121.253698:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845121.253751:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641874688 00000100:00100000:0.0:1788845121.253764:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641874688 00000100:00100000:0.0:1788845121.254019:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1696 00000100:00100000:0.0:1788845121.254034:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:18208:x1875739641874688:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:0.0:1788845121.254062:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845121.254085:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5be lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa949 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.254220:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af5c5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.254237:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af5c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.254249:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af5c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845121.254285:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788845121.254289:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845121.254308:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af5c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.254347:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5be lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa949 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.254365:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af5c5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa949 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.254373:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929ca00) 00010000:00010000:0.0:1788845121.254376:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788845121.254381:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006200, rc 0) 00000100:00100000:0.0:1788845121.254447:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f0358c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:18208:x1875739641874688:12345-192.168.201.29@tcp:101:rm.0 Request processed in 417us (756us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845121.254455:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1696 00010000:00010000:2.0:1788845121.254822:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.254840:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.254849:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.254861:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216fa00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.254866:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.254875:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.257966:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.259024:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641874816 00000100:00100000:3.0:1788845121.259034:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641874816 00000100:00100000:3.0:1788845121.259102:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1697 00000100:00100000:3.0:1788845121.259112:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a12680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:18208:x1875739641874816:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.259132:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.259151:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5cc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa950 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259212:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259226:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259236:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259273:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259281:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259288:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259389:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5e1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259396:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259404:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.259435:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.259452:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845121.259470:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb size from disk: 0 -> 234852 00000004:00010000:3.0:1788845121.259473:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb mtime from disk: 0 -> 1788844946 00000004:00010000:3.0:1788845121.259475:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb atime from disk: 0 -> 1788844946 00000004:00010000:3.0:1788845121.259478:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb ctime from disk: 0 -> 1788844946 00000004:00010000:3.0:1788845121.259481:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb blocks from disk: 18427282363563114435 -> 518 00002000:00010000:3.0:1788845121.259485:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 234852 00010000:00010000:3.0:1788845121.259490:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259497:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259504:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259510:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a0206c5000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.259515:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259522:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259534:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929ca00) 00010000:00010000:3.0:1788845121.259538:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.259542:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259559:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5000/0xb2187301287af5d3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259572:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259579:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259590:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259595:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a0206c5200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.259598:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259604:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259610:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xcf86d744 (ffff94a00be8d200) 00010000:00010000:3.0:1788845121.259612:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.259615:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259622:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c5200/0xb2187301287af5da lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259656:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af5cc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa950 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259669:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa950 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.259677:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:3.0:1788845121.259679:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788845121.259681:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:3.0:1788845121.259739:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a12680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641874816:12345-192.168.201.29@tcp:101:rm.0 Request processed in 627us (1782us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.259750:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1697 00010000:00010000:2.0:1788845121.259844:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x98980ae (ffff94a00be8d100) 00010000:00010000:2.0:1788845121.259853:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.259861:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.259872:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af5a9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.271063:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845121.271098:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641875072 00000100:00100000:0.0:1788845121.271110:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641875072 00000100:00100000:0.0:1788845121.271170:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1698 00000100:00100000:0.0:1788845121.271180:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:18208:x1875739641875072:12345-192.168.201.29@tcp:36:rm.0 00010000:00010000:0.0:1788845121.271233:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1788845121.271280:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216ee00/0xb2187301287af5e1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa950 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271299:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:0.0:1788845121.271315:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af5c5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa949 expref: 16 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271339:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929ca00) 00010000:00010000:0.0:1788845121.271346:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845121.271368:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271381:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271390:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271401:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271412:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271418:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271661:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271671:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.271677:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280075:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280379:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280395:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280406:0:11036:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280413:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280420:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280450:0:11036:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1788845121.280471:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280478:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280483:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280489:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.280494:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280502:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280515:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff94a01929c700) 00010000:00010000:0.0:1788845121.280518:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788845121.280522:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280529:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af5f6 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280547:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280553:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280563:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280568:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.280570:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280576:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280582:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929c400) 00010000:00010000:0.0:1788845121.280585:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.280587:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.280593:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af5e8 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788845121.280610:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000744fb1a5 reply state 000000008c59cc35 transno 4294967896 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845121.280620:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845121.280709:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:18208:x1875739641875072:12345-192.168.201.29@tcp:36:rm.0 Request processed in 9527us (9652us total) trans 4294967896 rc 0/0 00000100:00100000:0.0:1788845121.280719:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1698 00010000:00010000:1.0:1788845121.281339:0:5828:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788845121.281348:0:5828:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.281356:0:5828:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007600 already unused, calling callback (ffffffffc1717350) 00010000:00010000:1.0:1788845121.281360:0:5828:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.281368:0:5828:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.281384:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff94a01929c700) 00010000:00010000:1.0:1788845121.281387:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845121.281390:0:5828:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 1/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.281396:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af5fd lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.281414:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845121.281417:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845121.281419:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.288606:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.288622:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.288631:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.288637:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.288641:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.288650:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.288672:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xcf86d744 (ffff94a01929c900) 00010000:00010000:2.0:1788845121.288675:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.288679:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.288686:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af5ef lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.297276:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.297327:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641875200 00000100:00100000:3.0:1788845121.297337:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641875200 00000100:00100000:3.0:1788845121.297387:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1699 00000100:00100000:3.0:1788845121.297395:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a12d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+16:18208:x1875739641875200:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.297419:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.297440:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af604 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa957 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.297499:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af60b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.297512:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af60b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.297524:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af60b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.297566:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788845121.297570:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.297587:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af60b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.297609:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af604 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa957 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.297628:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af60b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa957 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.297648:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8d200) 00010000:00010000:3.0:1788845121.297651:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788845121.297654:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216e000, rc 0) 00000100:00100000:3.0:1788845121.297703:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a12d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:18208:x1875739641875200:12345-192.168.201.29@tcp:101:rm.0 Request processed in 308us (446us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.297710:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1699 00000100:00100000:0.0:1788845121.304167:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845121.304265:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641875328 00000100:00100000:0.0:1788845121.304274:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641875328 00000100:00100000:0.0:1788845121.304424:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1700 00000100:00100000:0.0:1788845121.304437:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:18208:x1875739641875328:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:0.0:1788845121.304459:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845121.304479:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af612 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa95e expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304538:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304561:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304574:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304640:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304648:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304655:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304733:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af627 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304741:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af627 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304748:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af627 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845121.304780:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7:0x0]: returning size 1049076 00000004:00200000:0.0:1788845121.304784:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845121.304800:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af627 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788845121.304816:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb size from disk: 0 -> 1048476 00000004:00010000:0.0:1788845121.304819:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb mtime from disk: 0 -> 1788844951 00000004:00010000:0.0:1788845121.304821:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb atime from disk: 0 -> 1788844951 00000004:00010000:0.0:1788845121.304824:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb ctime from disk: 0 -> 1788844951 00000004:00010000:0.0:1788845121.304826:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb blocks from disk: 18427282363563114435 -> 2054 00002000:00010000:0.0:1788845121.304830:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048476 00010000:00010000:0.0:1788845121.304834:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304852:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304860:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304866:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.304870:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.304877:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305064:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8d200) 00010000:00010000:0.0:1788845121.305067:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.305081:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305087:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af619 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305100:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305105:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305112:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305118:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.305120:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305126:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305133:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x32368eb3 (ffff94a01929c400) 00010000:00010000:0.0:1788845121.305135:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.305138:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305144:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006200/0xb2187301287af620 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305174:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006600/0xb2187301287af612 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa95e expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305187:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af627 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa95e expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.305265:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff94a01929ce00) 00010000:00010000:0.0:1788845121.305267:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:0.0:1788845121.305270:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007400, rc 0) 00000100:00100000:0.0:1788845121.305399:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641875328:12345-192.168.201.29@tcp:101:rm.0 Request processed in 965us (1242us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845121.305407:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1700 00000100:00100000:3.0:1788845121.309204:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.309237:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641875456 00000100:00100000:3.0:1788845121.309246:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641875456 00000100:00100000:3.0:1788845121.309294:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1281 00000100:00100000:3.0:1788845121.309444:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641875456:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.309462:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.309483:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa965 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845121.309507:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x3:0x0] initial LVB size: 1049076, mtime: 0x6a9f9b97, atime: 0x0, ctime: 0x6a9f9b97, blocks: 0x3 00010000:00010000:3.0:1788845121.309523:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=510s v=(0 0) 00010000:00010000:3.0:1788845121.309527:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788845121.309537:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.309564:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 7 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.309578:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a04216ee00, rc 0) 00000100:00100000:3.0:1788845121.309628:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641875456:12345-192.168.201.29@tcp:101:rm.0 Request processed in 326us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.309645:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1281 00000100:00080000:3.0:1788845121.309666:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845121.317222:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.317274:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641875712 00000100:00100000:2.0:1788845121.317284:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641875712 00000100:00100000:2.0:1788845121.317354:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1701 00000100:00100000:2.0:1788845121.317378:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641875712:12345-192.168.201.29@tcp:36:rm.0 00010000:00010000:2.0:1788845121.317439:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788845121.317490:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af627 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa95e expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317508:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff94a01929ce00) 00010000:00010000:2.0:1788845121.317526:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af60b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa957 expref: 17 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317537:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8d200) 00010000:00010000:2.0:1788845121.317543:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845121.317564:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317577:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317600:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317615:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317622:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317629:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317718:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317726:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.317732:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788845121.318585:0:5841:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].3, 64 off8384 00000004:00080000:2.0:1788845121.318597:0:5841:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 510515713566ns, next at 511515711038ns, rc = 0 00010000:00010000:2.0:1788845121.318655:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318666:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318674:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318685:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318692:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318698:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318715:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1788845121.318734:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318741:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318747:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318753:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.318758:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318765:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318776:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:2.0:1788845121.318779:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788845121.318783:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318793:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af643 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318810:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318816:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318823:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318828:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.318831:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318836:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318843:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6000) 00010000:00010000:2.0:1788845121.318845:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.318847:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.318853:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af635 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788845121.318867:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000125b3684 reply state 00000000c2bb296e transno 4294967897 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845121.318875:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845121.318979:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641875712:12345-192.168.201.29@tcp:36:rm.0 Request processed in 1612us (1763us total) trans 4294967897 rc 0/0 00000100:00100000:2.0:1788845121.318988:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1701 00010000:00010000:2.0:1788845121.319018:0:5827:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788845121.319024:0:5827:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.319030:0:5827:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864a00 already unused, calling callback (ffffffffc1717350) 00010000:00010000:2.0:1788845121.319033:0:5827:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.319040:0:5827:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.319050:0:5827:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:2.0:1788845121.319052:0:5827:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845121.319056:0:5827:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 1/0,0 mode: --/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.319062:0:5827:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af64a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.319073:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.319075:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.319077:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788845121.327557:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.327575:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.327582:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.327589:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.327594:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.327602:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.327617:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x32368eb3 (ffff94a0087c6800) 00010000:00010000:3.0:1788845121.327619:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.327624:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.327631:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af63c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845121.330274:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.330351:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641875840 00000100:00100000:3.0:1788845121.330359:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641875840 00000100:00100000:3.0:1788845121.330418:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1702 00000100:00100000:3.0:1788845121.330427:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:18208:x1875739641875840:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.330451:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.330470:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af651 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa96c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.330578:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af658 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.330592:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af658 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.330604:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af658 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.330637:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788845121.330641:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.330658:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af658 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.330680:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af651 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa96c expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.331232:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af658 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa96c expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.331243:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845121.331246:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788845121.331250:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c4200, rc 0) 00000100:00100000:3.0:1788845121.331316:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641875840:12345-192.168.201.29@tcp:101:rm.0 Request processed in 889us (1049us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.331334:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1702 00000100:00100000:3.0:1788845121.339589:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845121.339644:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641875968 00000100:00100000:1.0:1788845121.339654:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641875968 00000100:00100000:1.0:1788845121.339709:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1703 00000100:00100000:1.0:1788845121.339719:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641875968:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:1.0:1788845121.339740:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845121.339758:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af65f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa973 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.339816:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.339829:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.339840:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.339864:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.339872:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.339879:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.340047:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af674 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.340055:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af674 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.340063:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af674 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845121.340165:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8:0x0]: returning size 2097152 00000004:00200000:1.0:1788845121.340169:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.340452:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af674 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845121.340479:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb size from disk: 0 -> 1048576 00000004:00010000:3.0:1788845121.340483:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb mtime from disk: 0 -> 1788844958 00000004:00010000:3.0:1788845121.340486:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb atime from disk: 0 -> 1788844958 00000004:00010000:3.0:1788845121.340488:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb ctime from disk: 0 -> 1788844958 00000004:00010000:3.0:1788845121.340491:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb blocks from disk: 18427282363563114435 -> 2054 00002000:00010000:3.0:1788845121.340495:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788845121.340502:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340510:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340517:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340524:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.340529:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340537:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340550:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845121.340554:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.340558:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340565:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af666 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340580:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340587:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340592:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340597:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c0800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.340600:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340606:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340613:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xaa03ddaa (ffff94a019866a00) 00010000:00010000:3.0:1788845121.340615:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.340618:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340623:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af66d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340656:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af65f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa973 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340673:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af674 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa973 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.340687:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff94a019866e00) 00010000:00010000:3.0:1788845121.340689:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788845121.340692:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0000, rc 0) 00000100:00100000:2.0:1788845121.343459:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:18208:x1875739641875968:12345-192.168.201.29@tcp:101:rm.0 Request processed in 3731us (3869us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845121.343472:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1703 00000100:00100000:1.0:1788845121.347241:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.347510:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641876096 00000100:00100000:3.0:1788845121.347521:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641876096 00000100:00100000:3.0:1788845121.347578:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1282 00000100:00100000:3.0:1788845121.348276:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a034508850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641876096:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.348299:0:7983:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.348329:0:7983:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa97a expref: -99 pid: 7983 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788845121.348357:0:7983:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x3:0x0] initial LVB size: 2097152, mtime: 0x6a9f9b9e, atime: 0x0, ctime: 0x6a9f9b9e, blocks: 0x807 00010000:00010000:3.0:1788845121.349543:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=510s v=(0 0) 00010000:00010000:3.0:1788845121.349551:0:7983:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788845121.349565:0:7983:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.349585:0:7983:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 8 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.349598:0:7983:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a0206c5200, rc 0) 00000100:00100000:3.0:1788845121.349664:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a034508850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+8:2370:x1875739641876096:12345-192.168.201.29@tcp:101:rm.0 Request processed in 2075us (2429us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.349672:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1282 00000001:00080000:2.0:1788845121.356027:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967895 is committed 00000004:00080000:2.0:1788845121.356039:0:5861:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 2 committed 00000100:00100000:1.0:1788845121.357358:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845121.357393:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641876352 00000100:00100000:1.0:1788845121.357400:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641876352 00000100:00100000:3.0:1788845121.357425:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660447104:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1788845121.357448:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1704 00000100:00100000:1.0:1788845121.357459:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641876352:12345-192.168.201.29@tcp:36:rm.0 00000100:00100000:3.0:1788845121.357479:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1788845121.357505:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788845121.357543:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af674 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa973 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.357557:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660447104 00000100:00100000:3.0:1788845121.357562:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660447104 00010000:00010000:1.0:1788845121.357564:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff94a019866e00) 00010000:00010000:1.0:1788845121.357583:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a0206c4200/0xb2187301287af658 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa96c expref: 17 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.357595:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:1.0:1788845121.357601:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845121.357611:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1283 00000100:00100000:3.0:1788845121.357620:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660447104:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1788845121.357620:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.357638:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1788845121.357646:0:6682:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:2 count 1 00010000:00010000:1.0:1788845121.357648:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.357666:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.357668:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.357676:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.357691:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.357699:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.357710:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788845121.357715:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788845121.357720:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.357749:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 7 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.357769:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 7 pid: 7983 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.357771:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.357786:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.357793:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.357812:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035740000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6682:1875739660447232:192.168.201.29@tcp:104: 00000040:00080000:1.0:1788845121.358476:0:5840:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].3, 64 off8384 00010000:00010000:1.0:1788845121.358529:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358539:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358547:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358555:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358561:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358568:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358582:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1788845121.358597:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358603:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358610:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358616:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845121.358621:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358628:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358640:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff94a006406c00) 00010000:00010000:1.0:1788845121.358642:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788845121.358646:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358652:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af697 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358664:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358669:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358675:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358681:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845121.358683:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358689:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358695:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a006406600) 00010000:00010000:1.0:1788845121.358697:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845121.358699:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358705:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af682 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788845121.358722:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f6b92100 reply state 00000000822c84e9 transno 4294967898 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.358730:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845121.358769:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641876352:12345-192.168.201.29@tcp:36:rm.0 Request processed in 1312us (1415us total) trans 4294967898 rc 0/0 00000100:00100000:1.0:1788845121.358776:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1704 00010000:00010000:1.0:1788845121.358799:0:5828:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788845121.358805:0:5828:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358810:0:5828:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1200 already unused, calling callback (ffffffffc1717350) 00010000:00010000:1.0:1788845121.358814:0:5828:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358821:0:5828:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358829:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff94a006406c00) 00010000:00010000:1.0:1788845121.358831:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845121.358834:0:5828:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 1/0,0 mode: --/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358840:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af69e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.358850:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845121.358851:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845121.358853:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845121.358964:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff949f01e22240 going to sleep for 11 seconds 00000100:00080000:3.0:1788845121.358980:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788845121.361554:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845121.361624:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.361636:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.361643:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.361648:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c0a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.361653:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.361660:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.361674:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xaa03ddaa (ffff94a006406900) 00010000:00010000:0.0:1788845121.361677:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.361681:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.361687:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af689 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.361748:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641876480 00000100:00100000:1.0:1788845121.361757:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641876480 00010000:00010000:1.0:1788845121.361787:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 8 pid: 7983 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.361821:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 577 00000100:00100000:1.0:1788845121.361829:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641876480:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845121.361850:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845121.361856:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 8 pid: 7983 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.361873:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 8 pid: 7983 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.361915:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788845121.361920:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788845121.361925:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.361935:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.361952:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845121.361989:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+8:2369:x1875739641876480:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 161us (443us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845121.361996:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 577 00010000:00010000:3.0:1788845121.363163:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c3c34d0/0xb2187301287af59b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa942 expref: 7 pid: 7983 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.363182:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035740000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6682:1875739660447232:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845121.363265:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.363277:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.363285:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.363295:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.363302:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.363368:0:6682:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788845121.364379:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035741f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3312:x1875739660447104:12345-0@lo:6:osp-syn-0-0.0 Request processed in 6759us (6898us total) trans 4294967303 rc 0/0 00000100:00100000:3.0:1788845121.364390:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1283 00000100:00100000:3.0:1788845121.365957:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.366116:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641876608 00000100:00100000:2.0:1788845121.366125:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641876608 00000100:00100000:2.0:1788845121.366177:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1705 00000100:00100000:2.0:1788845121.366187:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035740000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+17:18208:x1875739641876608:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:2.0:1788845121.366371:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.366394:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af6a5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa981 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.366452:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af6ac lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.366465:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af6ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.366478:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af6ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845121.366510:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1788845121.366513:0:5841:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845121.366530:0:5841:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af6ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.366552:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af6a5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa981 expref: 19 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.366569:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af6ac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa981 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.366578:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:2.0:1788845121.366580:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788845121.366584:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00000100:00100000:2.0:1788845121.366634:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035740000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641876608:12345-192.168.201.29@tcp:101:rm.0 Request processed in 449us (683us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845121.366641:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1705 00000100:00100000:3.0:1788845121.368770:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:0.0:1788845121.368852:0:5827:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788845121.368861:0:5827:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.368873:0:5827:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a0206c5000 already unused, calling callback (ffffffffc1717350) 00010000:00010000:0.0:1788845121.368878:0:5827:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.368984:0:5827:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.369003:0:5827:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.369011:0:5827:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a075281/0xb2187301287af690 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845121.369102:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641876736 00000100:00100000:2.0:1788845121.369111:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641876736 00000100:00100000:2.0:1788845121.369157:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1706 00000100:00100000:2.0:1788845121.369165:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035742d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641876736:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:2.0:1788845121.369184:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.369383:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af6b3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa988 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369444:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369456:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369467:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369480:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369487:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369494:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369570:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af6c8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369579:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af6c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369585:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af6c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845121.369614:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845121.369629:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af6c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788845121.369646:0:5840:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb size from disk: 0 -> 10 00000004:00010000:2.0:1788845121.369649:0:5840:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb mtime from disk: 0 -> 1788844972 00000004:00010000:2.0:1788845121.369651:0:5840:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb atime from disk: 0 -> 1788844970 00000004:00010000:2.0:1788845121.369654:0:5840:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb ctime from disk: 0 -> 1788844972 00000004:00010000:2.0:1788845121.369656:0:5840:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb blocks from disk: 18427282363563114435 -> 10 00002000:00010000:2.0:1788845121.369660:0:5840:0:(mdt_io.c:1550:mdt_lvb2reply()) size 10 00010000:00010000:2.0:1788845121.369664:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369671:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369677:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369682:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.369686:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369692:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369701:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:2.0:1788845121.369704:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.369707:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369713:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af6ba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369722:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369728:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369734:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369739:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.369742:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369747:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369754:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5fca220a (ffff94a0087c6000) 00010000:00010000:2.0:1788845121.369756:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.369758:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369764:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af6c1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369792:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af6b3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa988 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369804:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af6c8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa988 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.369811:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff94a0087c6c00) 00010000:00010000:2.0:1788845121.369813:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:2.0:1788845121.369816:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864400, rc 0) 00000100:00100000:2.0:1788845121.369980:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035742d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:18208:x1875739641876736:12345-192.168.201.29@tcp:101:rm.0 Request processed in 815us (1214us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845121.369987:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1706 00010000:00010000:2.0:1788845121.370009:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.370012:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.370014:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1788845121.370289:0:3312:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff94a035741180/2, rc 0, transno 4294967303 00000100:00100000:1.0:1788845121.370307:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660447104:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1788845121.377085:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.377287:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641876992 00000100:00100000:3.0:1788845121.377295:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641876992 00000100:00100000:3.0:1788845121.377361:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1707 00000100:00100000:3.0:1788845121.377371:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:18208:x1875739641876992:12345-192.168.201.29@tcp:36:rm.0 00010000:00010000:3.0:1788845121.377425:0:5841:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788845121.377469:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af6c8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa988 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377486:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff94a0087c6c00) 00010000:00010000:3.0:1788845121.377501:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af6ac lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa981 expref: 18 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377510:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:3.0:1788845121.377515:0:5841:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845121.377533:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377545:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377554:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377565:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377573:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377579:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377663:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377671:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.377677:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378686:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378699:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378707:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378716:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378723:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378729:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378790:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788845121.378807:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378814:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378821:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378827:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216e800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.378832:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378849:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378858:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff94a00be8d400) 00010000:00010000:3.0:1788845121.378861:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788845121.378865:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 2/0,0 mode: --/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378871:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e800/0xb2187301287af6dd lrc: 0/0,0 mode: --/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378976:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378983:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378989:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.378994:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216fa00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.378998:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.379009:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.379017:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8db00) 00010000:00010000:3.0:1788845121.379020:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.379023:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.379029:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216fa00/0xb2187301287af6cf lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845121.379043:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000044c48c5f reply state 00000000ea7adba4 transno 4294967899 ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.379052:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845121.379205:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:18208:x1875739641876992:12345-192.168.201.29@tcp:36:rm.0 Request processed in 1822us (2125us total) trans 4294967899 rc 0/0 00000100:00100000:3.0:1788845121.379214:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1707 00010000:00010000:3.0:1788845121.382519:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.382534:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.382542:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.382548:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a04216e000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.382553:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.382565:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.382580:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5fca220a (ffff94a00be8d700) 00010000:00010000:3.0:1788845121.382583:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.382587:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.382593:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a04216e000/0xb2187301287af6d6 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845121.382723:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.382953:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641877120 00000100:00100000:3.0:1788845121.382962:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641877120 00000100:00100000:3.0:1788845121.383011:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1708 00000100:00100000:3.0:1788845121.383019:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641877120:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.383041:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.383058:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287af6eb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa98f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.383123:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af6f2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.383135:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af6f2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.383144:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af6f2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.383177:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788845121.383182:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.383207:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af6f2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.383228:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772800/0xb2187301287af6eb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa98f expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.383241:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af6f2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa98f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.383249:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845121.383251:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788845121.383262:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773200, rc 0) 00000100:00100000:3.0:1788845121.383305:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:18208:x1875739641877120:12345-192.168.201.29@tcp:101:rm.0 Request processed in 286us (586us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.383312:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1708 00010000:00010000:1.0:1788845121.384480:0:5828:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788845121.384490:0:5828:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.384498:0:5828:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773600 already unused, calling callback (ffffffffc1717350) 00010000:00010000:1.0:1788845121.384502:0:5828:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.384510:0:5828:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.384523:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff94a00be8d400) 00010000:00010000:1.0:1788845121.384525:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845121.384528:0:5828:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.384534:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773600/0xb2187301287af6e4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.384550:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845121.384552:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845121.384554:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845121.386249:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.386412:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641877248 00000100:00100000:3.0:1788845121.386421:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641877248 00000100:00100000:3.0:1788845121.386471:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1709 00000100:00100000:3.0:1788845121.386480:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:18208:x1875739641877248:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.386498:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.386512:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af6f9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa996 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386566:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386579:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386595:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386656:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386664:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386671:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386746:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af70e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386754:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af70e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386761:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af70e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.386790:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.386805:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af70e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845121.386823:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb size from disk: 0 -> 500 00000004:00010000:3.0:1788845121.386826:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb mtime from disk: 0 -> 1788844981 00000004:00010000:3.0:1788845121.386829:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb atime from disk: 0 -> 1788844980 00000004:00010000:3.0:1788845121.386831:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb ctime from disk: 0 -> 1788844981 00000004:00010000:3.0:1788845121.386834:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb blocks from disk: 18427282363563114435 -> 10 00002000:00010000:3.0:1788845121.386838:0:11036:0:(mdt_io.c:1550:mdt_lvb2reply()) size 500 00010000:00010000:3.0:1788845121.386843:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386850:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386856:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386863:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.386867:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386875:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386912:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845121.386915:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.386920:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386926:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af700 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386938:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386945:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386951:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386961:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.386964:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386970:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386977:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x584b25ba (ffff94a00be8d200) 00010000:00010000:3.0:1788845121.386979:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.386982:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.386988:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af707 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.387015:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af6f9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa996 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.387026:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af70e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa996 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.387034:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:3.0:1788845121.387037:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788845121.387038:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773e00, rc 0) 00000100:00100000:3.0:1788845121.387157:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:18208:x1875739641877248:12345-192.168.201.29@tcp:101:rm.0 Request processed in 677us (914us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.387165:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1709 00000100:00100000:0.0:1788845121.391137:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.391358:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641877504 00000100:00100000:3.0:1788845121.391368:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641877504 00000100:00100000:3.0:1788845121.391422:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1710 00000100:00100000:3.0:1788845121.391432:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:18208:x1875739641877504:12345-192.168.201.29@tcp:36:rm.0 00010000:00010000:3.0:1788845121.391480:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788845121.391520:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af70e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa996 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391538:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:3.0:1788845121.391554:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af6f2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa98f expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391563:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845121.391568:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845121.391586:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391599:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391609:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391619:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391626:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391633:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391717:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391725:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.391731:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392432:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392442:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392451:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392457:0:5839:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392464:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392470:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392484:0:5839:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788845121.392499:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392505:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392511:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392518:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.392522:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392529:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392537:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff94a00be8d000) 00010000:00010000:3.0:1788845121.392540:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788845121.392543:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392550:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af723 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392563:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392569:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392576:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392581:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.392583:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392593:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392599:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8d100) 00010000:00010000:3.0:1788845121.392601:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.392604:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.392609:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af715 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845121.392620:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000564716e3 reply state 00000000733f96c6 transno 4294967900 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.392629:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845121.392794:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:18208:x1875739641877504:12345-192.168.201.29@tcp:36:rm.0 Request processed in 1362us (1665us total) trans 4294967900 rc 0/0 00000100:00100000:3.0:1788845121.392802:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1710 00010000:00010000:0.0:1788845121.393010:0:5827:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788845121.393017:0:5827:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.393023:0:5827:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772400 already unused, calling callback (ffffffffc1717350) 00010000:00010000:0.0:1788845121.393027:0:5827:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.393033:0:5827:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.393043:0:5827:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff94a00be8d000) 00010000:00010000:0.0:1788845121.393045:0:5827:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788845121.393048:0:5827:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 1/0,0 mode: --/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.393054:0:5827:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772400/0xb2187301287af72a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.393074:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845121.393077:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845121.393079:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845121.396012:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.396027:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.396035:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.396041:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773400 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.396045:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.396052:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.396076:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x584b25ba (ffff94a00be8d800) 00010000:00010000:0.0:1788845121.396079:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.396083:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.396094:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773400/0xb2187301287af71c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845121.397116:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.397399:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641877632 00000100:00100000:3.0:1788845121.397408:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641877632 00000100:00100000:3.0:1788845121.397450:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1711 00000100:00100000:3.0:1788845121.397458:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:18208:x1875739641877632:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.397480:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.397497:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af731 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa99d expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.397553:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772000/0xb2187301287af738 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.397570:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772000/0xb2187301287af738 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.397580:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772000/0xb2187301287af738 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.397610:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788845121.397613:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.397630:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772000/0xb2187301287af738 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.397650:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af731 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa99d expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.397665:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772000/0xb2187301287af738 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa99d expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.397673:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8db00) 00010000:00010000:3.0:1788845121.397676:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788845121.397679:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772000, rc 0) 00000100:00100000:3.0:1788845121.397738:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:18208:x1875739641877632:12345-192.168.201.29@tcp:101:rm.0 Request processed in 280us (627us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.397745:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1711 00000100:00100000:1.0:1788845121.405797:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845121.405877:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641877888 00000100:00100000:0.0:1788845121.405920:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641877888 00000100:00100000:0.0:1788845121.405967:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 45 00000100:00100000:0.0:1788845121.405976:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00777c050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+20:18208:x1875739641877888:12345-192.168.201.29@tcp:35:rm.0 00000100:00100000:0.0:1788845121.406562:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00777c050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:18208:x1875739641877888:12345-192.168.201.29@tcp:35:rm.0 Request processed in 585us (772us total) trans 4294967901 rc 0/0 00000100:00100000:0.0:1788845121.406571:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 45 00000100:00100000:0.0:1788845121.409460:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845121.409539:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641878144 00000100:00100000:1.0:1788845121.409546:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641878144 00000100:00100000:1.0:1788845121.409593:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1712 00000100:00100000:1.0:1788845121.409601:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:18208:x1875739641878144:12345-192.168.201.29@tcp:36:rm.0 00010000:00010000:1.0:1788845121.409650:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 5 locks, starting at 0 00010000:00010000:1.0:1788845121.409673:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865600/0xb2187301287aa2a5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb:0x0].0x0 bits 0xd/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6223 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409703:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864c00/0xb2187301287aa2c8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6238 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409714:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772600/0xb2187301287aa2ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109a622a expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409727:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a031e6b800/0xb2187301287aa24a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x19/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000010000 nid: 192.168.201.29@tcp remote: 0x3100577d109a6200 expref: 18 pid: 5840 timeout: 477 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409737:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff94a019866000) 00010000:00010000:1.0:1788845121.409750:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772000/0xb2187301287af738 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa99d expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409755:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8db00) 00010000:00010000:1.0:1788845121.409759:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845121.409778:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409788:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409794:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409802:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409806:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409810:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409966:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409976:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.409983:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.410742:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.410753:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.410760:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.410767:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.410772:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.410777:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.410969:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1788845121.410989:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.410995:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411001:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411005:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c0800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845121.411008:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411015:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411023:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff94a006406f00) 00010000:00010000:1.0:1788845121.411025:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788845121.411029:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411034:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af74d lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411047:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411053:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411058:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411061:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845121.411064:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411173:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411182:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a006406c00) 00010000:00010000:1.0:1788845121.411184:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845121.411187:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.411192:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af73f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788845121.411208:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007bf76d65 reply state 00000000ebc6dff3 transno 4294967902 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.411216:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845121.411297:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:18208:x1875739641878144:12345-192.168.201.29@tcp:36:rm.0 Request processed in 1692us (1841us total) trans 4294967902 rc 0/0 00000100:00100000:1.0:1788845121.411305:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1712 00010000:00010000:2.0:1788845121.412155:0:5828:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788845121.412164:0:5828:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.412172:0:5828:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1000 already unused, calling callback (ffffffffc1717350) 00010000:00010000:2.0:1788845121.412175:0:5828:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.412182:0:5828:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.412245:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff94a006406f00) 00010000:00010000:2.0:1788845121.412248:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788845121.412252:0:5828:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.412360:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af754 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.412378:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.412380:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.412382:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.413789:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.413801:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.413808:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.413814:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c0000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.413818:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.413826:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.413837:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x248233be (ffff94a006406600) 00010000:00010000:2.0:1788845121.413840:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.413844:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.413855:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0000/0xb2187301287af746 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845121.463388:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.463516:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641878272 00000100:00100000:3.0:1788845121.463525:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641878272 00000100:00100000:3.0:1788845121.463578:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1713 00000100:00100000:3.0:1788845121.463587:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:18211:x1875739641878272:12345-192.168.201.29@tcp:101:mkdir.0 00010000:00010000:3.0:1788845121.463609:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.463632:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af75b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9a4 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.463705:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af762 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.463718:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af762 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.463730:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af762 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.463763:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788845121.463767:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.463784:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af762 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.463805:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af75b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9a4 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.463822:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af762 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9a4 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.463831:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8d100) 00010000:00010000:3.0:1788845121.463834:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788845121.463838:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773000, rc 0) 00000100:00100000:3.0:1788845121.463946:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:18211:x1875739641878272:12345-192.168.201.29@tcp:101:mkdir.0 Request processed in 359us (567us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.463953:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1713 00000100:00100000:3.0:1788845121.477080:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.477122:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845121.477216:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641878400 00000100:00100000:1.0:1788845121.477226:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641878400 00000100:00100000:1.0:1788845121.477280:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641878656 00000100:00100000:1.0:1788845121.477283:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641878656 00000100:00100000:1.0:1788845121.477312:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1714 00000100:00100000:1.0:1788845121.477330:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2367:x1875739641878400:12345-192.168.201.29@tcp:41:mkdir.0 00000004:00000024:1.0:1788845121.477373:0:5840:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 148373504 pending 0 free 2258370560 avail 2256273408 00000100:00100000:1.0:1788845121.477436:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01fc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2367:x1875739641878400:12345-192.168.201.29@tcp:41:mkdir.0 Request processed in 108us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845121.477447:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1714 00000100:00100000:1.0:1788845121.477463:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1715 00000100:00100000:1.0:1788845121.477469:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:18211:x1875739641878656:12345-192.168.201.29@tcp:36:mkdir.0 00010000:00010000:1.0:1788845121.477505:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845121.477528:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773000/0xb2187301287af762 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9a4 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.477546:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8d100) 00010000:00010000:1.0:1788845121.477556:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845121.477621:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.477633:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.477643:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.477691:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.477698:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.477705:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845121.478409:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00010000:1.0:1788845121.478419:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.478427:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.478435:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.478441:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845121.478445:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.478452:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.478460:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a006406c00) 00010000:00010000:1.0:1788845121.478463:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845121.478467:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.478473:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af769 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788845121.478486:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000698889d4 reply state 000000007348512b transno 4294967903 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.478495:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845121.478544:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a019d90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:18211:x1875739641878656:12345-192.168.201.29@tcp:36:mkdir.0 Request processed in 1074us (1420us total) trans 4294967903 rc 0/0 00000100:00100000:1.0:1788845121.478551:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1715 00010000:00010000:0.0:1788845121.493564:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.493580:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.493588:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.493594:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c0800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.493599:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.493606:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.493627:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff94a006406800) 00010000:00010000:0.0:1788845121.493630:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.493634:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.493640:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0800/0xb2187301287af770 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.511961:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.512101:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641878784 00000100:00100000:2.0:1788845121.512110:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641878784 00000100:00100000:2.0:1788845121.512164:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1716 00000100:00100000:2.0:1788845121.512175:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:18212:x1875739641878784:12345-192.168.201.29@tcp:101:chmod.0 00010000:00010000:2.0:1788845121.512198:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.512219:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af777 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9ab expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.512291:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af77e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.512312:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af77e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.512333:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af77e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845121.512366:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1788845121.512369:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845121.512386:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af77e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.512408:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af777 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ab expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.512425:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af77e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ab expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.512434:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6000) 00010000:00010000:2.0:1788845121.512437:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788845121.512440:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864000, rc 0) 00000100:00100000:2.0:1788845121.512495:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:18212:x1875739641878784:12345-192.168.201.29@tcp:101:chmod.0 Request processed in 322us (542us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845121.512502:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1716 00000100:00100000:3.0:1788845121.517269:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.517404:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641879040 00000100:00100000:2.0:1788845121.517413:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641879040 00000100:00100000:2.0:1788845121.517467:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1717 00000100:00100000:2.0:1788845121.517476:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:18212:x1875739641879040:12345-192.168.201.29@tcp:36:chmod.0 00010000:00010000:2.0:1788845121.517522:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845121.517545:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af77e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ab expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.517563:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6000) 00010000:00010000:2.0:1788845121.517573:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845121.517592:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.517604:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.517614:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788845121.517863:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000705ea65b reply state 00000000e078ac96 transno 4294967904 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845121.517877:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1788845121.517915:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000100:00100000:2.0:1788845121.517980:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03bea8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:18212:x1875739641879040:12345-192.168.201.29@tcp:36:chmod.0 Request processed in 505us (728us total) trans 4294967904 rc 0/0 00000100:00100000:2.0:1788845121.517988:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1717 00010000:00010000:2.0:1788845121.520531:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.520545:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.520553:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.520559:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.520563:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.520571:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.520586:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6700) 00010000:00010000:2.0:1788845121.520589:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788845121.520593:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.520599:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af785 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.556304:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.556811:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641879168 00000100:00100000:2.0:1788845121.556821:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641879168 00000100:00100000:2.0:1788845121.556927:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1718 00000100:00100000:2.0:1788845121.556950:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:18213:x1875739641879168:12345-192.168.201.29@tcp:101:chown.0 00010000:00010000:2.0:1788845121.556974:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.556994:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af78c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9b2 expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.557055:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af793 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.557078:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af793 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.557089:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af793 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845121.557121:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1788845121.557125:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845121.557145:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af793 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.557168:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af78c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9b2 expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.557185:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af793 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9b2 expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.557194:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6700) 00010000:00010000:2.0:1788845121.557196:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788845121.557200:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864000, rc 0) 00000100:00100000:2.0:1788845121.557266:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:18213:x1875739641879168:12345-192.168.201.29@tcp:101:chown.0 Request processed in 330us (971us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845121.557273:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1718 00000100:00100000:1.0:1788845121.560758:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.560872:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641879296 00000100:00100000:2.0:1788845121.560907:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641879296 00000100:00100000:2.0:1788845121.560950:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1719 00000100:00100000:2.0:1788845121.560958:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:18213:x1875739641879296:12345-192.168.201.29@tcp:101:chown.0 00010000:00010000:2.0:1788845121.560976:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.560994:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af79a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9b9 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561052:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561380:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561394:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561409:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561416:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561423:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561501:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af7af lrc: 3/1,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561509:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af7af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561516:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af7af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845121.561544:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00000004:00200000:2.0:1788845121.561547:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845121.561562:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af7af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561574:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561580:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561587:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561592:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.561597:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561604:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561616:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6700) 00010000:00010000:2.0:1788845121.561618:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.561622:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561633:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af7a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561644:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561650:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561656:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561661:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.561664:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561670:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561676:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff94a0087c6000) 00010000:00010000:2.0:1788845121.561679:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.561681:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561687:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af7a8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561713:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af79a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9b9 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561725:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af7af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9b9 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.561732:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:2.0:1788845121.561735:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788845121.561737:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865e00, rc 0) 00000100:00100000:2.0:1788845121.561795:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:18213:x1875739641879296:12345-192.168.201.29@tcp:101:chown.0 Request processed in 838us (1046us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845121.561803:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1719 00000100:00100000:1.0:1788845121.565823:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.566084:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641879552 00000100:00100000:2.0:1788845121.566095:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641879552 00000100:00100000:2.0:1788845121.566146:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1720 00000100:00100000:2.0:1788845121.566155:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:18213:x1875739641879552:12345-192.168.201.29@tcp:36:chown.0 00010000:00010000:2.0:1788845121.566649:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788845121.566683:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af7af lrc: 0/0,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9b9 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.566702:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:2.0:1788845121.566711:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845121.566732:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.566744:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.566753:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788845121.621576:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967901 is committed 00000001:00080000:3.0:1788845121.621594:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967903 is committed 00000001:00080000:3.0:1788845121.621604:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967905 is committed 00000004:00080000:3.0:1788845121.621612:0:5861:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 4 committed 00000100:00100000:0.0:1788845121.621770:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788845121.621789:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01bb3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660447360:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1788845121.621851:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845121.622050:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660447360 00000100:00100000:0.0:1788845121.622057:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660447360 00000100:00100000:0.0:1788845121.622171:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1284 00000100:00100000:0.0:1788845121.622181:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0360b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660447360:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1788845121.622269:0:6684:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:3 count 1 00010000:00010000:0.0:1788845121.622292:0:6684:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 3/0,1 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.622327:0:6684:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.622337:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788845121.622342:0:6684:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788845121.622347:0:6684:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 4/0,1 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.622380:0:6684:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 6 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.622398:0:6684:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 6 pid: 6682 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845121.622410:0:6684:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a00891b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660447488:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845121.622462:0:6684:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d20000 going to sleep for 11 seconds 00000100:00080000:0.0:1788845121.622474:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00010000:2.0:1788845121.624614:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000013883c3c reply state 00000000ce2282e7 transno 4294967905 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845121.624634:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1788845121.624647:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00080000:2.0:1788845121.624717:0:5840:0:(ldlm_lib.c:3482:target_send_reply()) Schedule reply immediately 00000100:00100000:2.0:1788845121.624740:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+12:18213:x1875739641879552:12345-192.168.201.29@tcp:36:chown.0 Request processed in 58582us (58921us total) trans 4294967905 rc 0/0 00000100:00100000:2.0:1788845121.624749:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1720 00010000:00010000:0.0:1788845121.625479:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.625490:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.625497:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.625503:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.625507:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.625515:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.625530:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a0087c6c00) 00010000:00010000:0.0:1788845121.625533:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788845121.625538:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.625544:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af7b6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845121.635434:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845121.636826:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641879680 00000100:00100000:1.0:1788845121.636845:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641879680 00010000:00010000:1.0:1788845121.636921:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 7 pid: 6682 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.636954:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 578 00000100:00100000:1.0:1788845121.636964:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641879680:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845121.636977:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845121.636983:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 7 pid: 6682 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.637001:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 7 pid: 6682 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.637026:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788845121.637030:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788845121.637036:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.637046:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.637181:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845121.637311:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+7:2370:x1875739641879680:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 348us (1886us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845121.637328:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 578 00010000:00010000:0.0:1788845121.638840:0:6684:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e67e0ed/0xb2187301287af62e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa965 expref: 6 pid: 6682 timeout: 610 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845121.638862:0:6684:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a00891b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6684:1875739660447488:192.168.201.29@tcp:104: 00010000:00010000:0.0:1788845121.638904:0:6684:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.638915:0:6684:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.638923:0:6684:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.638934:0:6684:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.638941:0:6684:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.638965:0:6684:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788845121.639520:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0360b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3314:x1875739660447360:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17339us (17667us total) trans 4294967523 rc 0/0 00000100:00100000:0.0:1788845121.639532:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1284 00000004:00080000:1.0:1788845121.639625:0:3314:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff949f01bb3800/2, rc 0, transno 4294967523 00000100:00100000:1.0:1788845121.639631:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01bb3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660447360:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1788845121.639854:0:5827:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788845121.639862:0:5827:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.639871:0:5827:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007000 already unused, calling callback (ffffffffc1717350) 00010000:00010000:2.0:1788845121.639906:0:5827:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.639924:0:5827:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.639941:0:5827:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 1/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.639949:0:5827:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000293e28df/0xb2187301287af7bd lrc: 0/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6684 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.639968:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.639970:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.639972:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845121.646527:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.646868:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641879808 00000100:00100000:3.0:1788845121.646910:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641879808 00000100:00100000:3.0:1788845121.646961:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1721 00000100:00100000:3.0:1788845121.646969:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:18214:x1875739641879808:12345-192.168.201.29@tcp:101:touch.500 00010000:00010000:3.0:1788845121.646991:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.647009:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af7c4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9c0 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647077:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647090:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647101:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647114:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647122:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647128:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647418:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af7d9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647427:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af7d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647434:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af7d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.647452:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00000004:00200000:3.0:1788845121.647455:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845121.647472:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af7d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647483:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647490:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647497:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647503:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.647506:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647514:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647525:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6700) 00010000:00010000:3.0:1788845121.647528:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.647531:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647538:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af7cb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647549:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647555:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647560:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647565:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845121.647568:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647574:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647580:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff94a00be8da00) 00010000:00010000:3.0:1788845121.647582:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845121.647584:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647591:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af7d2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647617:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af7c4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c0 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647634:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af7d9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c0 expref: 12 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.647641:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:3.0:1788845121.647644:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1788845121.647646:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773e00, rc 0) 00000100:00100000:3.0:1788845121.647699:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00891b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+12:18214:x1875739641879808:12345-192.168.201.29@tcp:101:touch.500 Request processed in 730us (1182us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.647705:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1721 00000100:00100000:3.0:1788845121.651639:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.651726:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641880064 00000100:00100000:2.0:1788845121.651735:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641880064 00000100:00100000:2.0:1788845121.651788:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1722 00000100:00100000:2.0:1788845121.651797:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+12:18214:x1875739641880064:12345-192.168.201.29@tcp:101:touch.500 00010000:00010000:2.0:1788845121.651816:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.651842:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af7e0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9c7 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.652475:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.652490:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.652501:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.652556:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 3/0,1 mode: --/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.652564:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.652571:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653466:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653477:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653485:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788845121.653519:0:5841:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:0.0:1788845121.653522:0:5841:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb mtime from disk: 0 -> 1788845121 00000004:00010000:0.0:1788845121.653525:0:5841:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb atime from disk: 0 -> 1788845121 00000004:00010000:0.0:1788845121.653527:0:5841:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb ctime from disk: 0 -> 1788845121 00000004:00010000:0.0:1788845121.653530:0:5841:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb blocks from disk: 18427282363563114435 -> 2 00002000:00010000:0.0:1788845121.653534:0:5841:0:(mdt_io.c:1550:mdt_lvb2reply()) size 0 00010000:00010000:0.0:1788845121.653538:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653545:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653552:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653558:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.653562:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653569:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653579:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:0.0:1788845121.653582:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.653585:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653592:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006a00/0xb2187301287af7e7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788845121.653603:0:5841:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c9823ef1 reply state 000000003e946183 transno 4294967906 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845121.653611:0:5841:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1788845121.653638:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af7e0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 15 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653652:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.653659:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff94a01929c300) 00010000:00010000:0.0:1788845121.653662:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:0.0:1788845121.653664:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006e00, rc 0) 00000100:00100000:0.0:1788845121.653734:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:18214:x1875739641880064:12345-192.168.201.29@tcp:101:touch.500 Request processed in 1938us (2103us total) trans 4294967906 rc 0/0 00000100:00100000:0.0:1788845121.653743:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1722 00010000:00010000:2.0:1788845121.660110:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.660126:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.660133:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.660139:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.660144:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.660151:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.660168:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0xc8586294 (ffff94a01929c400) 00010000:00010000:2.0:1788845121.660172:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.660177:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 2/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.660184:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007000/0xb2187301287af7ee lrc: 0/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.660851:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845121.661408:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641880320 00000100:00100000:0.0:1788845121.661418:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641880320 00000100:00100000:0.0:1788845121.661473:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1723 00000100:00100000:0.0:1788845121.661482:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:18214:x1875739641880320:12345-192.168.201.29@tcp:36:touch.500 00010000:00010000:0.0:1788845121.661569:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.661584:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.661593:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.661625:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 14 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.661642:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 14 pid: 5841 timeout: 610 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845121.661655:0:5839:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660447616:192.168.201.29@tcp:104: 00000100:00100000:0.0:1788845121.661726:0:5839:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a008d209c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788845121.664944:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845121.665045:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641880448 00000100:00100000:1.0:1788845121.665053:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641880448 00010000:00010000:1.0:1788845121.665097:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 15 pid: 5841 timeout: 610 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.665133:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 579 00000100:00100000:1.0:1788845121.665141:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aa9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641880448:12345-192.168.201.29@tcp:102:ldlm_bl.0 00010000:00010000:1.0:1788845121.665157:0:5824:0:(ldlm_lockd.c:1655:ldlm_handle_convert0()) ### server-side convert handler START ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 15 pid: 5841 timeout: 610 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.665172:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 15 pid: 5841 timeout: 610 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.665183:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x59/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 15 pid: 5841 timeout: 610 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.665191:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff94a01929c300) 00010000:00010000:1.0:1788845121.665203:0:5824:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:1.0:1788845121.665206:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.665216:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.665222:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.665249:0:5824:0:(ldlm_lockd.c:1704:ldlm_handle_convert0()) ### server-side convert handler END, rc = 0 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x59/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 15 pid: 5841 timeout: 610 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.665311:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aa9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641880448:12345-192.168.201.29@tcp:102:ldlm_bl.0 Request processed in 169us (375us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845121.665327:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 579 00000100:00100000:3.0:1788845121.665847:0:5839:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5839:1875739660447616:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845121.665870:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.666008:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.666014:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845121.666150:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:3.0:1788845121.666153:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb mtime from disk: 1788845121 -> 1788845121 00000004:00010000:3.0:1788845121.666156:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb atime from disk: 1788845121 -> 1788845121 00000004:00010000:3.0:1788845121.666158:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb ctime from disk: 1788845121 -> 1788845121 00000004:00010000:3.0:1788845121.666161:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb blocks from disk: 2 -> 2 00000004:00010000:3.0:1788845121.666168:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000064a00dec reply state 00000000351415fe transno 4294967907 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845121.666176:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845121.666438:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:18214:x1875739641880320:12345-192.168.201.29@tcp:36:touch.500 Request processed in 4956us (5590us total) trans 4294967907 rc 0/0 00000100:00100000:3.0:1788845121.666445:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1723 00000100:00100000:0.0:1788845121.669705:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00010000:00010000:2.0:1788845121.669773:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.669785:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.669792:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.669798:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.669804:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.669812:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.669827:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff94a01929c300) 00010000:00010000:2.0:1788845121.669830:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.669844:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.669851:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006000/0xb2187301287af803 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.670217:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641880704 00000100:00100000:1.0:1788845121.670226:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641880704 00000100:00100000:1.0:1788845121.670283:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 46 00000100:00100000:1.0:1788845121.670292:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:18214:x1875739641880704:12345-192.168.201.29@tcp:35:touch.500 00000100:00100000:1.0:1788845121.670526:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:18214:x1875739641880704:12345-192.168.201.29@tcp:35:touch.500 Request processed in 235us (829us total) trans 4294967908 rc 0/0 00000100:00100000:1.0:1788845121.670533:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 46 00000100:00100000:1.0:1788845121.680564:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.680657:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641880832 00000100:00100000:2.0:1788845121.680667:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641880832 00000100:00100000:2.0:1788845121.680720:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1724 00000100:00100000:2.0:1788845121.680729:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:18215:x1875739641880832:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:2.0:1788845121.680752:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845121.680771:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af80a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9ce expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.680833:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af811 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.680857:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af811 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.680867:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af811 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845121.680933:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00000004:00200000:2.0:1788845121.680936:0:5840:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845121.680953:0:5840:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af811 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.680978:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af80a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ce expref: 18 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.680995:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af811 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ce expref: 17 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.681003:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:2.0:1788845121.681006:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788845121.681009:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864a00, rc 0) 00000100:00100000:2.0:1788845121.681058:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+17:18215:x1875739641880832:12345-192.168.201.29@tcp:101:rm.0 Request processed in 330us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845121.681238:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1724 00000100:00100000:2.0:1788845121.685020:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.685099:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641881088 00000100:00100000:3.0:1788845121.685108:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641881088 00000100:00100000:3.0:1788845121.685151:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1725 00000100:00100000:3.0:1788845121.685160:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:18215:x1875739641881088:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.685179:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.685206:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af818 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9d5 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.685291:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00010000:3.0:1788845121.685609:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af818 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9d5 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.685622:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6700) 00010000:00010000:3.0:1788845121.685625:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788845121.685630:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af818 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9d5 expref: 19 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.685643:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773a00, rc 0) 00000100:00100000:3.0:1788845121.685702:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:18215:x1875739641881088:12345-192.168.201.29@tcp:101:rm.0 Request processed in 543us (690us total) trans 4294967909 rc 301/301 00000100:00100000:3.0:1788845121.685709:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1725 00000100:00100000:1.0:1788845121.693975:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.694057:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641881216 00000100:00100000:2.0:1788845121.694075:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641881216 00000100:00100000:2.0:1788845121.694116:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 47 00000100:00100000:2.0:1788845121.694126:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:18215:x1875739641881216:12345-192.168.201.29@tcp:37:rm.0 00000100:00100000:2.0:1788845121.698633:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+18:18215:x1875739641881216:12345-192.168.201.29@tcp:37:rm.0 Request processed in 4499us (4658us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845121.698645:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 47 00000100:00100000:2.0:1788845121.702206:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845121.702501:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641881472 00000100:00100000:0.0:1788845121.702510:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641881472 00000100:00100000:0.0:1788845121.702592:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 48 00000100:00100000:0.0:1788845121.702603:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:18215:x1875739641881472:12345-192.168.201.29@tcp:35:rm.0 00000100:00100000:0.0:1788845121.702811:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:69859fee-4322-4bed-b50b-7a298318b0f6+19:18215:x1875739641881472:12345-192.168.201.29@tcp:35:rm.0 Request processed in 210us (622us total) trans 4294967910 rc 0/0 00000100:00100000:0.0:1788845121.702819:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 48 00000100:00100000:1.0:1788845121.705723:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.705846:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641881728 00000100:00100000:3.0:1788845121.705856:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641881728 00000100:00100000:3.0:1788845121.705930:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1726 00000100:00100000:3.0:1788845121.705940:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:18215:x1875739641881728:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845121.705962:0:5841:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845121.705980:0:5841:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af826 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9dc expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.706072:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af82d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.706085:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af82d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x11:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.706096:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af82d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x11:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845121.706106:0:5841:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00010000:3.0:1788845121.706462:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af826 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9dc expref: 22 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.706480:0:5841:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af82d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x11:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9dc expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.706489:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:3.0:1788845121.706492:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1788845121.706495:0:5841:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773200, rc 0) 00000100:00100000:3.0:1788845121.706550:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:18215:x1875739641881728:12345-192.168.201.29@tcp:101:rm.0 Request processed in 612us (836us total) trans 4294967911 rc 0/0 00000100:00100000:3.0:1788845121.706561:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1726 00000100:00100000:2.0:1788845121.708907:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.709022:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641881856 00000100:00100000:3.0:1788845121.709030:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641881856 00000100:00100000:3.0:1788845121.709082:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1727 00000100:00100000:3.0:1788845121.709091:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f98000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:18215:x1875739641881856:12345-192.168.201.29@tcp:41:rm.0 00000004:00000024:3.0:1788845121.709143:0:11036:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 148373504 pending 0 free 7716470784 avail 7712276480 00000100:00100000:3.0:1788845121.709305:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f98000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:18215:x1875739641881856:12345-192.168.201.29@tcp:41:rm.0 Request processed in 215us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845121.709322:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1727 00000100:00100000:1.0:1788845121.711825:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.711916:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641882112 00000100:00100000:2.0:1788845121.711925:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641882112 00000100:00100000:2.0:1788845121.711971:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1728 00000100:00100000:2.0:1788845121.711979:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:18215:x1875739641882112:12345-192.168.201.29@tcp:36:rm.0 00010000:00010000:2.0:1788845121.712026:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788845121.712077:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006e00/0xb2187301287af7f5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x59/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c7 expref: 21 pid: 5841 timeout: 610 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712097:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff94a01929c300) 00010000:00010000:2.0:1788845121.712115:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af811 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ce expref: 20 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712124:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:2.0:1788845121.712127:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788845121.712142:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712155:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712168:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712183:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712200:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712207:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712305:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712341:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712348:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712733:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712742:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712749:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712757:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712768:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712774:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712799:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1788845121.712814:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712820:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712826:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712837:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.712841:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712849:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712856:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:2.0:1788845121.712858:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788845121.712862:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712868:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864200/0xb2187301287af849 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712910:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712918:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712924:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712929:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.712932:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712939:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712945:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:2.0:1788845121.712947:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.712951:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.712957:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af83b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788845121.712966:0:5840:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001a07f54e reply state 0000000075875842 transno 4294967912 ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788845121.712974:0:5840:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788845121.713043:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:18215:x1875739641882112:12345-192.168.201.29@tcp:36:rm.0 Request processed in 1063us (1224us total) trans 4294967912 rc 0/0 00000100:00100000:2.0:1788845121.713050:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1728 00010000:00010000:1.0:1788845121.713401:0:5828:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788845121.713409:0:5828:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.713416:0:5828:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865e00 already unused, calling callback (ffffffffc1717350) 00010000:00010000:1.0:1788845121.713420:0:5828:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.713427:0:5828:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.713437:0:5828:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff94a0087c6d00) 00010000:00010000:1.0:1788845121.713439:0:5828:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788845121.713442:0:5828:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.713449:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865e00/0xb2187301287af850 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845121.713463:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845121.713465:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788845121.713466:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845121.715591:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.715609:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.715616:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.715622:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.715627:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.715635:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.715647:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0xc8586294 (ffff94a0087c6300) 00010000:00010000:2.0:1788845121.715650:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.715653:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.715660:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865200/0xb2187301287af842 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0xc8586294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845121.717933:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845121.718050:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641882368 00000100:00100000:2.0:1788845121.718059:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641882368 00000100:00100000:2.0:1788845121.718110:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 49 00000100:00100000:2.0:1788845121.718119:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:18215:x1875739641882368:12345-192.168.201.29@tcp:35:rm.0 00000100:00100000:2.0:1788845121.718365:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+21:18215:x1875739641882368:12345-192.168.201.29@tcp:35:rm.0 Request processed in 246us (438us total) trans 4294967913 rc 0/0 00000100:00100000:2.0:1788845121.718373:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 49 00000100:00100000:1.0:1788845121.720652:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845121.720721:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641882624 00000100:00100000:3.0:1788845121.720729:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641882624 00000100:00100000:3.0:1788845121.720774:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1729 00000100:00100000:3.0:1788845121.720782:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03998000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:18215:x1875739641882624:12345-192.168.201.29@tcp:36:rm.0 00010000:00010000:3.0:1788845121.720829:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1788845121.720851:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af82d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x11:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9dc expref: 21 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.720871:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af7d9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9c0 expref: 20 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.721092:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a00be8d300) 00010000:00010000:3.0:1788845121.721109:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864000/0xb2187301287af793 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9b2 expref: 19 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845121.721118:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6700) 00010000:00010000:3.0:1788845121.721125:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788845121.721268:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.721281:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.721291:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.721303:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.721310:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.721325:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.721535:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.721544:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.721551:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845121.722045:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00010000:0.0:1788845121.722124:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722133:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722140:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722147:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.722150:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722157:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722167:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff94a01929c200) 00010000:00010000:0.0:1788845121.722169:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788845121.722173:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722184:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af865 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722203:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722210:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722216:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722220:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845121.722223:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722229:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722236:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929ca00) 00010000:00010000:0.0:1788845121.722238:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845121.722241:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845121.722247:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af857 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788845121.722256:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009c9df573 reply state 00000000b1a0bdfc transno 4294967914 ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845121.722264:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788845121.722347:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03998000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:18215:x1875739641882624:12345-192.168.201.29@tcp:36:rm.0 Request processed in 1564us (1700us total) trans 4294967914 rc 0/0 00000100:00100000:0.0:1788845121.722356:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1729 00010000:00010000:2.0:1788845121.724862:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.724919:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.724927:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.724933:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007800 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845121.724938:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.724945:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.724959:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff94a01929c100) 00010000:00010000:2.0:1788845121.724962:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845121.724967:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845121.724973:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af85e lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845122.130098:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:1.0:1788845122.308573:0:6708:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 69859fee-4322-4bed-b50b-7a298318b0f6 00000001:00080000:1.0:1788845122.308593:0:6708:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967303 is committed 00000100:00100000:0.0:1788845122.772335:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a17480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660448128:0@lo:400:kworker.0 00000100:00100000:0.0:1788845122.772410:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845122.772480:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1788845122.772487:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f023dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660447744:0@lo:400:kworker.0 00000100:00100000:0.0:1788845122.772518:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845122.772543:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660448256:0@lo:400:kworker.0 00000100:00100000:0.0:1788845122.772569:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845122.772598:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660447744 00000100:00100000:0.0:1788845122.772604:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660447744 00000100:00100000:0.0:1788845122.772646:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:0.0:1788845122.772656:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660447744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788845122.772703:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:05b4ec57-2769-4797-9b3d-21582f14799e+19:3313:x1875739660447744:12345-0@lo:400:kworker.0 Request processed in 48us (184us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845122.772711:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:0.0:1788845122.772757:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f023dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:05b4ec57-2769-4797-9b3d-21582f14799e:3313:1875739660447744:0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773110:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660447872:0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773144:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845122.773191:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a17800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660448384:0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773214:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845122.773236:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788845122.773242:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660448000:0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773265:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845122.773317:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660448256 00000100:00100000:2.0:1788845122.773321:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660448256 00000100:00100000:2.0:1788845122.773357:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660447872 00000100:00100000:2.0:1788845122.773361:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660447872 00000100:00100000:2.0:1788845122.773373:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660448000 00000100:00100000:2.0:1788845122.773375:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660448000 00000100:00100000:2.0:1788845122.773391:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:2.0:1788845122.773398:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660448256:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773440:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3313:x1875739660448256:12345-0@lo:400:kworker.0 Request processed in 42us (870us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845122.773446:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:2.0:1788845122.773460:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:2.0:1788845122.773466:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660447872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773497:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f99180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660447872:12345-0@lo:400:kworker.0 Request processed in 31us (352us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845122.773503:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:2.0:1788845122.773514:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:2.0:1788845122.773521:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660448000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773562:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a02da4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1875739660448000:12345-0@lo:400:kworker.0 Request processed in 41us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845122.773567:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:2.0:1788845122.773589:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660448128 00000100:00100000:2.0:1788845122.773593:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660448128 00000100:00100000:2.0:1788845122.773618:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660448384 00000100:00100000:2.0:1788845122.773621:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660448384 00000100:00100000:2.0:1788845122.773637:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1788845122.773643:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660448128:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773670:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660448128:12345-0@lo:400:kworker.0 Request processed in 27us (1261us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845122.773675:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1788845122.773684:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1788845122.773689:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3311:x1875739660448384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773709:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3311:x1875739660448384:12345-0@lo:400:kworker.0 Request processed in 19us (494us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845122.773713:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00080000:2.0:1788845122.773730:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845122.773753:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3311:1875739660448000:0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773763:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f02367b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660447872:0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773772:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a17800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660448384:0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773797:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3313:1875739660448256:0@lo:400:kworker.0 00000100:00100000:2.0:1788845122.773805:0:3313:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967303 gen 2 00000004:00080000:2.0:1788845122.773809:0:3313:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff94a035741180, transno 4294967303 00000100:00100000:2.0:1788845122.773818:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a17480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660448128:0@lo:400:kworker.0 00010000:00010000:1.0:1788845123.154219:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845123.282107:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a14380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660448512:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788845123.282182:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845123.282356:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660448512 00000100:00100000:0.0:1788845123.282365:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660448512 00000100:00100000:0.0:1788845123.282433:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:0.0:1788845123.282444:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aeb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3312:x1875739660448512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788845123.282479:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788845123.282523:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aeb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3312:x1875739660448512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (340us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845123.282530:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:1.0:1788845123.282941:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a14380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660448512:0@lo:13:osp-pre-1-0.0 00000100:00080000:3.0:1788845123.283374:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1788845124.178639:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1788845124.306112:0:6803:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 513503225933ns, next at 514503223575ns, rc = 0 00000100:00100000:1.0:1788845124.306202:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788845124.306227:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a010481c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660448640:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788845124.306321:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845124.308603:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660448640 00000100:00100000:0.0:1788845124.308616:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660448640 00000100:00100000:0.0:1788845124.308692:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:0.0:1788845124.308705:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660448640:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788845124.308745:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1788845124.308833:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660448640:12345-0@lo:13:osp-pre-0-0.0 Request processed in 132us (2511us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845124.308840:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00080000:3.0:1788845124.309797:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845124.309954:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a010481c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660448640:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788845124.389987:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845124.390039:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845124.390119:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641883008 00000100:00100000:3.0:1788845124.390130:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641883008 00000100:00100000:3.0:1788845124.390230:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 177 00000100:00100000:3.0:1788845124.390242:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641883008:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845124.390319:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641883008:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 79us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845124.390326:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 177 00000100:00080000:0.0:1788845124.390392:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845124.390659:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641882752 00000100:00100000:2.0:1788845124.390664:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641882752 00000100:00100000:2.0:1788845124.390701:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 327 00000100:00100000:2.0:1788845124.390708:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2370:x1875739641882752:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845124.390738:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2370:x1875739641882752:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 31us (763us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845124.390744:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 327 00000100:00100000:1.0:1788845124.407059:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845124.407105:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845124.407203:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641882880 00000100:00100000:2.0:1788845124.407214:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641882880 00000100:00100000:2.0:1788845124.407312:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 243 00000100:00100000:2.0:1788845124.407323:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03ae9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641882880:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845124.407398:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03ae9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+19:2370:x1875739641882880:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 77us (348us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845124.407406:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 243 00000100:00080000:0.0:1788845124.407478:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788845124.407862:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641883136 00000100:00100000:3.0:1788845124.407868:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641883136 00000100:00100000:3.0:1788845124.407945:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 178 00000100:00100000:3.0:1788845124.407953:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03aeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641883136:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845124.407996:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03aeb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641883136:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 44us (890us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845124.408002:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 178 00000100:00080000:0.0:1788845124.408058:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1788845125.202626:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1788845126.006548:0:7641:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967523 is committed 00010000:00010000:2.0:1788845126.226067:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788845126.730465:0:15892:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity test 36a: MDS utime check (mknod, utime) ======= 01:25:24 (1788845124) 00000100:00100000:3.0:1788845126.855165:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845126.856559:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641883264 00000100:00100000:0.0:1788845126.856572:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641883264 00000100:00100000:0.0:1788845126.856655:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1730 00000100:00100000:0.0:1788845126.856665:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:18587:x1875739641883264:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:0.0:1788845126.856691:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845126.856709:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af86c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9e3 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.856780:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af873 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.856795:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af873 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.856809:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af873 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845126.856840:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788845126.856844:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845126.856865:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af873 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.856933:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007c00/0xb2187301287af86c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9e3 expref: 21 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.856951:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af873 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9e3 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.856957:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929ca00) 00010000:00010000:0.0:1788845126.856959:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788845126.856962:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845126.857024:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:18587:x1875739641883264:12345-192.168.201.29@tcp:101:rm.0 Request processed in 361us (1869us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845126.857031:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1730 00000100:00080000:3.0:1788845126.857758:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845126.873770:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845126.873854:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641883392 00000100:00100000:3.0:1788845126.873863:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641883392 00000100:00100000:3.0:1788845126.873971:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1731 00000100:00100000:3.0:1788845126.873984:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010480700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:18587:x1875739641883392:12345-192.168.201.29@tcp:101:rm.0 00010000:00010000:3.0:1788845126.874004:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845126.874022:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af87a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9ea expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874089:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874103:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874368:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874386:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874394:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874401:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874476:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874487:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874493:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874499:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845126.874503:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874514:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874526:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929ca00) 00010000:00010000:3.0:1788845126.874529:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845126.874533:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874539:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af881 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874551:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874556:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874561:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874566:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845126.874570:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874577:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874586:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff94a00be8d300) 00010000:00010000:3.0:1788845126.874589:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845126.874591:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874601:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af888 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874634:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af87a lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ea expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874645:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929ca00) 00010000:00010000:3.0:1788845126.874647:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845126.874650:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773e00/0xb2187301287af87a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ea expref: 21 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.874658:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773e00, rc 0) 00000100:00100000:3.0:1788845126.885330:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010480700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+20:18587:x1875739641883392:12345-192.168.201.29@tcp:101:rm.0 Request processed in 11337us (11556us total) trans 0 rc 301/301 00000100:00100000:3.0:1788845126.885343:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1731 00000100:00100000:3.0:1788845126.926010:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845126.926426:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641883648 00000100:00100000:1.0:1788845126.926436:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641883648 00000100:00100000:1.0:1788845126.926491:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1732 00000100:00100000:1.0:1788845126.926502:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+20:18590:x1875739641883648:12345-192.168.201.29@tcp:36:utime.0 00010000:00010000:1.0:1788845126.926558:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845126.926585:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af873 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9e3 expref: 20 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.926604:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929ca00) 00010000:00010000:1.0:1788845126.926615:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845126.926844:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.926858:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.926868:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.926902:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.926910:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.926918:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.929010:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.929025:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.929033:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.929039:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1000 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845126.929042:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.929049:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.929064:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a006406200) 00010000:00010000:1.0:1788845126.929067:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845126.929071:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.929077:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af88f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788845126.929096:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000db20cdf1 reply state 0000000061797343 transno 4294967915 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845126.929337:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845126.929456:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:18590:x1875739641883648:12345-192.168.201.29@tcp:36:utime.0 Request processed in 2953us (3452us total) trans 4294967915 rc 0/0 00000100:00100000:1.0:1788845126.929464:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1732 00000100:00100000:0.0:1788845126.947771:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845126.947807:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641883776 00000100:00100000:0.0:1788845126.947816:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641883776 00000100:00100000:0.0:1788845126.947871:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1733 00000100:00100000:0.0:1788845126.947907:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:18590:x1875739641883776:12345-192.168.201.29@tcp:101:utime.0 00010000:00010000:0.0:1788845126.947931:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845126.947953:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af89d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9f1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.948017:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af8a4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.948036:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af8a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.948048:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af8a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845126.948082:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788845126.948086:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845126.948114:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af8a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.948139:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af89d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9f1 expref: 23 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.948156:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af8a4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9f1 expref: 22 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.948165:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929c200) 00010000:00010000:0.0:1788845126.948168:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788845126.948172:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02006800, rc 0) 00000100:00100000:0.0:1788845126.948219:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a010481c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+22:18590:x1875739641883776:12345-192.168.201.29@tcp:101:utime.0 Request processed in 339us (457us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845126.948238:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1733 00010000:00010000:3.0:1788845126.948623:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.948632:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.948638:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.948644:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845126.948648:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.948661:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.948672:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff94a006406400) 00010000:00010000:3.0:1788845126.948674:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845126.948678:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.948684:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1600/0xb2187301287af896 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845126.956264:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845126.957062:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641883904 00000100:00100000:0.0:1788845126.957072:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641883904 00000100:00100000:0.0:1788845126.957125:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1734 00000100:00100000:0.0:1788845126.957135:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:18590:x1875739641883904:12345-192.168.201.29@tcp:101:utime.0 00010000:00010000:0.0:1788845126.957154:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845126.957172:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af8ab lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9f8 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957243:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957259:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957269:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957293:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957301:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957307:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957392:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af8c0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957400:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af8c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957407:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af8c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845126.957435:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 0 00000004:00200000:0.0:1788845126.957439:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845126.957454:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af8c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957465:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af8c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957472:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957479:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957486:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957491:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02006c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845126.957495:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957503:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957514:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929c200) 00010000:00010000:0.0:1788845126.957516:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845126.957520:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957526:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006c00/0xb2187301287af8b2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957537:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957548:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957554:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957558:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f02007600 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:0.0:1788845126.957561:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957567:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957572:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff94a01929cd00) 00010000:00010000:0.0:1788845126.957574:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788845126.957577:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957582:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007600/0xb2187301287af8b9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957608:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007400/0xb2187301287af8ab lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9f8 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957620:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af8c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9f8 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845126.957653:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a01929cf00) 00010000:00010000:0.0:1788845126.957656:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788845126.957659:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007200, rc 0) 00000100:00100000:0.0:1788845126.957715:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f023dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:18590:x1875739641883904:12345-192.168.201.29@tcp:101:utime.0 Request processed in 581us (1460us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845126.957721:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1734 00000004:00080000:3.0:1788845126.962089:0:5861:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 3 committed 00000001:00080000:3.0:1788845126.962115:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967914 is committed 00000100:00100000:1.0:1788845126.962250:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660448768:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1788845126.962385:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845126.962636:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660448768 00000100:00100000:3.0:1788845126.962644:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660448768 00000100:00100000:3.0:1788845126.962703:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1285 00000100:00100000:3.0:1788845126.962713:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660448768:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1788845126.962748:0:6682:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:3 count 1 00010000:00010000:3.0:1788845126.962772:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.962795:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 5 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.962804:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=6s v=(0 0) 00010000:00010000:3.0:1788845126.962808:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788845126.962813:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.962847:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 5 pid: 7983 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.962866:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 5 pid: 7983 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845126.962879:0:6682:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03a15180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6682:1875739660448896:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845126.962954:0:6682:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1000 going to sleep for 11 seconds 00000100:00080000:0.0:1788845126.963192:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788845126.990052:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845126.990145:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845126.990191:0:6682:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03a15180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6682:1875739660448896:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845126.990216:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845126.990256:0:5826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641884288 00000100:00100000:3.0:1788845126.990262:0:5826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641884288 00010000:00010000:3.0:1788845126.990312:0:5826:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 6 pid: 7983 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845126.990344:0:5826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 580 00000100:00100000:3.0:1788845126.990352:0:5826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2370:x1875739641884288:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788845126.990364:0:5826:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845126.990370:0:5826:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 6 pid: 7983 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990386:0:5826:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 6 pid: 7983 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990408:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca2b623e/0xb2187301287af67b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa97a expref: 6 pid: 7983 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990421:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788845126.990425:0:5826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788845126.990430:0:5826:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990439:0:5826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990458:0:5826:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788845126.990499:0:5826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:69859fee-4322-4bed-b50b-7a298318b0f6+5:2370:x1875739641884288:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 148us (353us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845126.990507:0:5826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 580 00010000:00010000:3.0:1788845126.990530:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990538:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990544:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990550:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845126.990566:0:6682:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1788845126.990588:0:5828:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788845126.990595:0:5828:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845126.990603:0:5828:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773e00 already unused, calling callback (ffffffffc1717350) 00010000:00010000:2.0:1788845126.990607:0:5828:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845126.990616:0:5828:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845126.990627:0:5828:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845126.990633:0:5828:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bfbc83a/0xb2187301287af8c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845126.990650:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845126.990652:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788845126.990653:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788845126.991452:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788845126.994615:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641884160 00000100:00100000:1.0:1788845126.994629:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641884160 00000100:00100000:1.0:1788845126.994691:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1735 00000100:00100000:1.0:1788845126.994701:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:18590:x1875739641884160:12345-192.168.201.29@tcp:36:utime.0 00010000:00010000:1.0:1788845126.994757:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845126.994784:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007200/0xb2187301287af8c0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9f8 expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.994801:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a01929cf00) 00010000:00010000:1.0:1788845126.994814:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788845126.994860:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.994874:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845126.994904:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788845126.995040:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003f29bbf1 reply state 00000000be55b9cf transno 4294967916 ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845126.995049:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788845126.995152:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:18590:x1875739641884160:12345-192.168.201.29@tcp:36:utime.0 Request processed in 452us (5108us total) trans 4294967916 rc 0/0 00000100:00100000:1.0:1788845126.995159:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1735 00000100:00100000:3.0:1788845126.996764:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03444000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3314:x1875739660448768:12345-0@lo:6:osp-syn-0-0.0 Request processed in 34046us (34374us total) trans 4294967304 rc 0/0 00000100:00100000:3.0:1788845126.996777:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1285 00000004:00080000:3.0:1788845126.996969:0:3314:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff94a035741180/2, rc 0, transno 4294967304 00000100:00100000:3.0:1788845126.996976:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035741180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660448768:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1788845127.003856:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845127.003874:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845127.004024:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845127.004033:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845127.004038:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845127.004046:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845127.004062:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a006406200) 00010000:00010000:2.0:1788845127.004065:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845127.004070:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845127.004076:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ce lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845127.006694:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845127.007013:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641884416 00000100:00100000:1.0:1788845127.007029:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641884416 00000100:00100000:1.0:1788845127.007101:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1736 00000100:00100000:1.0:1788845127.007112:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:18590:x1875739641884416:12345-192.168.201.29@tcp:101:utime.0 00010000:00010000:1.0:1788845127.007137:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845127.007158:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aa9ff expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.007232:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.007248:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.007259:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a006406d00) 00010000:00010000:1.0:1788845127.007262:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788845127.007266:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c0600, rc 0) 00000100:00100000:1.0:1788845127.007337:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f656300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:18590:x1875739641884416:12345-192.168.201.29@tcp:101:utime.0 Request processed in 226us (655us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845127.007348:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1736 00000100:00100000:0.0:1788845127.020824:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845127.021187:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641884544 00000100:00100000:2.0:1788845127.021203:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641884544 00000100:00100000:2.0:1788845127.021342:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 50 00000100:00100000:2.0:1788845127.021355:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00777f450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:18590:x1875739641884544:12345-192.168.201.29@tcp:49:utime.0 00000100:00100000:2.0:1788845127.021814:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00777f450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+14:18590:x1875739641884544:12345-192.168.201.29@tcp:49:utime.0 Request processed in 462us (1003us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845127.021822:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 50 00000100:00100000:3.0:1788845127.043735:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845127.043842:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641884672 00000100:00100000:1.0:1788845127.043855:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641884672 00000100:00100000:1.0:1788845127.043964:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1737 00000100:00100000:1.0:1788845127.043973:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03a15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:18590:x1875739641884672:12345-192.168.201.29@tcp:101:utime.0 00010000:00010000:1.0:1788845127.043995:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845127.044011:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af8dc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aaa06 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044065:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044076:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044085:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044095:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044100:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044103:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044173:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044177:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044181:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788845127.044205:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 0 00000004:00200000:1.0:1788845127.044207:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788845127.044232:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044243:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044255:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.044265:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046758:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046770:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c0e00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845127.046774:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046783:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046802:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929c200) 00010000:00010000:1.0:1788845127.046806:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845127.046810:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046817:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0e00/0xb2187301287af8e3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046836:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046841:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046847:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046852:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff949f036c1200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:1.0:1788845127.046855:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046861:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046868:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff94a006406300) 00010000:00010000:1.0:1788845127.046870:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788845127.046872:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.046877:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1200/0xb2187301287af8ea lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.047028:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0a00/0xb2187301287af8dc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 16 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.047044:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 15 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845127.047053:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a006406d00) 00010000:00010000:1.0:1788845127.047055:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788845127.047057:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1000, rc 0) 00000100:00100000:2.0:1788845127.047421:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03a15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:18590:x1875739641884672:12345-192.168.201.29@tcp:101:utime.0 Request processed in 3445us (3704us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845127.047430:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1737 00010000:00010000:2.0:1788845127.250057:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1788845127.535830:0:6708:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967304 is committed 00000100:00100000:2.0:1788845127.890600:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:05b4ec57-2769-4797-9b3d-21582f14799e:3311:1875739660449024:0@lo:400:kworker.0 00000100:00100000:0.0:1788845127.890627:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660449280:0@lo:400:kworker.0 00000100:00100000:0.0:1788845127.890688:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845127.890689:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845127.890760:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f775880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3311:1875739660449536:0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.890788:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845127.890807:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788845127.890812:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660449152:0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.890833:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845127.890850:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f775f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660449664:0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.890872:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845127.890917:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660449024 00000100:00100000:2.0:1788845127.890923:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660449024 00000100:00100000:2.0:1788845127.890967:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:2.0:1788845127.890976:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3311:x1875739660449024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891020:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:05b4ec57-2769-4797-9b3d-21582f14799e+19:3311:x1875739660449024:12345-0@lo:400:kworker.0 Request processed in 45us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845127.891027:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:2.0:1788845127.891073:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:05b4ec57-2769-4797-9b3d-21582f14799e:3311:1875739660449024:0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891121:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660449280 00000100:00100000:2.0:1788845127.891125:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660449280 00000100:00100000:2.0:1788845127.891155:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660449536 00000100:00100000:2.0:1788845127.891158:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660449536 00000100:00100000:2.0:1788845127.891169:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660449152 00000100:00100000:2.0:1788845127.891171:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660449152 00000100:00100000:2.0:1788845127.891184:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:2.0:1788845127.891190:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660449280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891234:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660449280:12345-0@lo:400:kworker.0 Request processed in 45us (550us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845127.891240:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:2.0:1788845127.891252:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:2.0:1788845127.891270:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1875739660449536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891302:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1875739660449536:12345-0@lo:400:kworker.0 Request processed in 44us (513us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845127.891307:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:2.0:1788845127.891319:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1788845127.891325:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0345d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660449152:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891363:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0345d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1875739660449152:12345-0@lo:400:kworker.0 Request processed in 38us (529us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845127.891369:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1788845127.891388:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660449408:0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891412:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845127.891447:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660449664 00000100:00100000:2.0:1788845127.891451:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660449664 00000100:00100000:2.0:1788845127.891475:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660449408 00000100:00100000:2.0:1788845127.891478:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660449408 00000100:00100000:2.0:1788845127.891494:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:2.0:1788845127.891500:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01f9b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660449664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891533:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01f9b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1875739660449664:12345-0@lo:400:kworker.0 Request processed in 33us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845127.891539:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:2.0:1788845127.891550:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1788845127.891556:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660449408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891585:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660449408:12345-0@lo:400:kworker.0 Request processed in 29us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845127.891591:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1788845127.891617:0:3314:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967304 gen 2 00000004:00080000:2.0:1788845127.891622:0:3314:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff94a035741180, transno 4294967304 00000100:00100000:2.0:1788845127.891632:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660449408:0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891844:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f654000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3311:1875739660449152:0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891853:0:3311:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967523 gen 2 00000004:00080000:2.0:1788845127.891857:0:3311:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff949f01bb3800, transno 4294967523 00000100:00100000:2.0:1788845127.891868:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f775f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3311:1875739660449664:0@lo:400:kworker.0 00000100:00100000:2.0:1788845127.891879:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f775880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3311:1875739660449536:0@lo:400:kworker.0 00000100:00080000:2.0:1788845127.891941:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788845127.891946:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845127.900240:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660449280:0@lo:400:kworker.0 00010000:00010000:2.0:1788845128.274152:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845128.402474:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a009687100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660449792:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845128.402550:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845128.402813:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660449792 00000100:00100000:3.0:1788845128.402822:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660449792 00000100:00100000:3.0:1788845128.402878:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:3.0:1788845128.402915:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660449792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788845128.402969:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788845128.403024:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660449792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 113us (473us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845128.403031:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:0.0:1788845128.403700:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a009687100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660449792:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1788845129.300819:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845129.426469:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03445c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660449920:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788845129.426543:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845129.426717:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660449920 00000100:00100000:1.0:1788845129.426725:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660449920 00000100:00100000:1.0:1788845129.426784:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:1.0:1788845129.426795:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660449920:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788845129.426829:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788845129.426877:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660449920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845129.426910:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00080000:2.0:1788845129.427048:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845129.427096:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03445c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660449920:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845129.499529:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845129.499574:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845129.499609:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845129.499679:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641884800 00000100:00100000:0.0:1788845129.499690:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641884800 00000100:00100000:1.0:1788845129.499774:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641885056 00000100:00100000:1.0:1788845129.499779:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641885056 00000100:00100000:3.0:1788845129.503639:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845129.503780:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641885184 00000100:00100000:3.0:1788845129.503791:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641885184 00000100:00100000:2.0:1788845129.504063:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641884928 00000100:00100000:2.0:1788845129.504069:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641884928 00000100:00100000:0.0:1788845129.508923:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 329 00000100:00100000:0.0:1788845129.508951:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2369:x1875739641884800:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845129.509041:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2369:x1875739641884800:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 105us (9524us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845129.509060:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 329 00000100:00100000:1.0:1788845129.518602:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 181 00000100:00100000:1.0:1788845129.518618:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641885056:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845129.518705:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:69859fee-4322-4bed-b50b-7a298318b0f6+5:2369:x1875739641885056:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 80us (19118us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845129.518713:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 181 00000100:00100000:3.0:1788845129.518931:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 182 00000100:00100000:3.0:1788845129.518941:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641885184:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:3.0:1788845129.518970:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:69859fee-4322-4bed-b50b-7a298318b0f6+6:2369:x1875739641885184:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 32us (15342us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845129.518976:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 182 00000100:00080000:3.0:1788845129.520992:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788845129.521001:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788845129.521004:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845129.529580:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 247 00000100:00100000:2.0:1788845129.529597:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a009687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641884928:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845129.530232:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a009687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:69859fee-4322-4bed-b50b-7a298318b0f6+15:2369:x1875739641884928:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 103us (30084us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845129.530244:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 247 00010000:00010000:2.0:1788845130.323602:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788845131.346042:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1788845132.097753:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967916 is committed 00010000:00010000:2.0:1788845132.370350:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788845133.013698:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:05b4ec57-2769-4797-9b3d-21582f14799e:3311:1875739660450048:0@lo:400:kworker.0 00000100:00100000:1.0:1788845133.013739:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f03412680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660450176:0@lo:400:kworker.0 00000100:00100000:3.0:1788845133.016250:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660450304:0@lo:400:kworker.0 00000100:00100000:1.0:1788845133.017825:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845133.018339:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660450176 00000100:00100000:0.0:1788845133.018351:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660450176 00000100:00100000:0.0:1788845133.018419:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:1.0:1788845133.018420:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660450432:0@lo:400:kworker.0 00000100:00100000:0.0:1788845133.018435:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03445c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660450176:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845133.018496:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845133.018509:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03445c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660450176:12345-0@lo:400:kworker.0 Request processed in 77us (690us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845133.018518:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:1.0:1788845133.018568:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660450432 00000100:00100000:0.0:1788845133.018570:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660450688:0@lo:400:kworker.0 00000100:00100000:1.0:1788845133.018575:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660450432 00000100:00100000:0.0:1788845133.018603:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788845133.018649:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03412680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660450176:0@lo:400:kworker.0 00000100:00100000:0.0:1788845133.018670:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1788845133.018675:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035929880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3312:1875739660450560:0@lo:400:kworker.0 00000100:00100000:0.0:1788845133.018700:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845133.021932:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845133.022017:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660450560 00000100:00100000:2.0:1788845133.022029:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660450560 00000100:00100000:2.0:1788845133.022406:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:2.0:1788845133.022420:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02382680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1875739660450560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845133.022528:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02382680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1875739660450560:12345-0@lo:400:kworker.0 Request processed in 113us (3827us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845133.022537:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:3.0:1788845133.024484:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788845133.024546:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660450304 00000100:00100000:3.0:1788845133.024555:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660450304 00000100:00100000:3.0:1788845133.024652:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1788845133.024664:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660450304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845133.024678:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660450048 00000100:00100000:2.0:1788845133.024685:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660450048 00000100:00100000:2.0:1788845133.024725:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:3.0:1788845133.024727:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660450304:12345-0@lo:400:kworker.0 Request processed in 64us (249us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845133.024736:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3311:x1875739660450048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788845133.024749:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1788845133.024798:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660450304:0@lo:400:kworker.0 00000100:00100000:2.0:1788845133.024959:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3311:x1875739660450048:12345-0@lo:400:kworker.0 Request processed in 225us (3037us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845133.024967:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00080000:2.0:1788845133.025129:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1788845133.025135:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845133.025167:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3312:1875739660450560:0@lo:400:kworker.0 00000100:00100000:3.0:1788845133.025343:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f03410700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:05b4ec57-2769-4797-9b3d-21582f14799e:3311:1875739660450048:0@lo:400:kworker.0 00000100:00100000:1.0:1788845133.026492:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660450688 00000100:00100000:1.0:1788845133.026503:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660450688 00000100:00100000:1.0:1788845133.026594:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1788845133.026606:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660450432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845133.026706:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660450432:12345-0@lo:400:kworker.0 Request processed in 103us (8210us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845133.026714:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1788845133.026728:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1788845133.026734:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660450688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845133.026768:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660450688:12345-0@lo:400:kworker.0 Request processed in 34us (8165us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845133.026775:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1788845133.026824:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035929c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660450432:0@lo:400:kworker.0 00000100:00100000:1.0:1788845133.026858:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660450688:0@lo:400:kworker.0 00000100:00080000:2.0:1788845133.027076:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1788845133.395401:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845133.545485:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788845133.545521:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f655880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660450816:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788845133.545635:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845133.546001:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660450816 00000100:00100000:2.0:1788845133.546012:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660450816 00000100:00100000:2.0:1788845133.546081:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:2.0:1788845133.546092:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660450816:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845133.546153:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788845133.546227:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660450816:12345-0@lo:13:osp-pre-1-0.0 Request processed in 125us (581us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845133.546236:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:1.0:1788845133.546603:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f655880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660450816:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1788845134.417999:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845134.553657:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788845134.553678:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660450944:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845134.553741:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845134.553829:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660450944 00000100:00100000:1.0:1788845134.553837:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660450944 00000100:00100000:1.0:1788845134.553923:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:1.0:1788845134.553932:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660450944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788845134.553984:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788845134.554031:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01d8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660450944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 101us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845134.554038:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:1.0:1788845134.554075:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660450944:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788845134.619763:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845134.619799:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845134.619827:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845134.619856:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845134.619899:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641885440 00000100:00100000:0.0:1788845134.619908:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641885440 00000100:00100000:0.0:1788845134.619982:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 251 00000100:00100000:0.0:1788845134.619992:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02380380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2368:x1875739641885440:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845134.620059:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02380380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:2368:x1875739641885440:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 69us (231us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845134.620066:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 251 00000100:00100000:0.0:1788845134.620103:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641885568 00000100:00100000:0.0:1788845134.620107:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641885568 00000100:00100000:0.0:1788845134.620130:0:7983:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641885696 00000100:00100000:0.0:1788845134.620132:0:7983:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641885696 00000100:00100000:0.0:1788845134.620149:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 185 00000100:00100000:0.0:1788845134.620154:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02381f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641885568:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845134.620180:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02381f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+5:2368:x1875739641885568:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 26us (381us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845134.620186:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 185 00000100:00100000:0.0:1788845134.620197:0:7983:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 186 00000100:00100000:0.0:1788845134.620203:0:7983:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641885696:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:0.0:1788845134.620221:0:7983:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:69859fee-4322-4bed-b50b-7a298318b0f6+6:2368:x1875739641885696:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 19us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845134.620226:0:7983:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 186 00000100:00080000:0.0:1788845134.620240:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788845134.620243:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1788845134.620245:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788845134.621879:0:5834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641885312 00000100:00100000:2.0:1788845134.621908:0:5834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641885312 00000100:00100000:2.0:1788845134.621968:0:5834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 331 00000100:00100000:2.0:1788845134.621977:0:5834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a0096d2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641885312:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845134.622052:0:5834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a0096d2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2368:x1875739641885312:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 76us (2298us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845134.622061:0:5834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 331 00010000:00010000:2.0:1788845135.442099:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1788845136.339655:0:16092:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity test 36b: OST utime check (open, utime) ======== 01:25:34 (1788845134) 00000100:00100000:0.0:1788845136.399758:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845136.399855:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641885952 00000100:00100000:3.0:1788845136.399865:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641885952 00000100:00100000:3.0:1788845136.399956:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1738 00000100:00100000:3.0:1788845136.399967:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+13:19024:x1875739641885952:12345-192.168.201.29@tcp:101:bash.0 00010000:00010000:3.0:1788845136.399988:0:5840:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845136.399992:0:5840:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788845136.400019:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02006800/0xb2187301287af8a4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9f1 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.400037:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929c200) 00010000:00010000:3.0:1788845136.400049:0:5840:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845136.400076:0:5840:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af8f8 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aaa0d expref: -99 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.400171:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.400185:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.400193:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.400201:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 5/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.400233:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 13 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.400249:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 13 pid: 5839 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845136.400263:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660451072:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845136.400349:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 13 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.400359:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 13 pid: 11036 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845136.400368:0:5840:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660451200:192.168.201.29@tcp:104: 00000100:00100000:3.0:1788845136.400390:0:5840:0:(client.c:2715:ptlrpc_set_wait()) set ffff94a0360b1600 going to sleep for 11 seconds 00000100:00080000:0.0:1788845136.400445:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788845136.430765:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845136.430809:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845136.430925:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641886080 00000100:00100000:1.0:1788845136.430939:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641886080 00010000:00010000:1.0:1788845136.430993:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 14 pid: 11036 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845136.431030:0:13304:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641886208 00000100:00100000:1.0:1788845136.431034:0:13304:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641886208 00010000:00010000:1.0:1788845136.431047:0:13304:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 15 pid: 5839 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788845136.431081:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 581 00000100:00100000:1.0:1788845136.431090:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641886080:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845136.431103:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845136.431110:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 15 pid: 11036 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.431122:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 15 pid: 11036 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.431145:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a006406d00) 00010000:00010000:1.0:1788845136.431151:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.431159:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845136.431210:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03410700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641886080:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 120us (455us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845136.431218:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 581 00000100:00100000:1.0:1788845136.431235:0:13304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 582 00000100:00100000:1.0:1788845136.431242:0:13304:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641886208:12345-192.168.201.29@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788845136.431250:0:13304:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845136.431254:0:13304:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 14 pid: 5839 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.431270:0:13304:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 14 pid: 5839 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.431286:0:13304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a006406d00) 00010000:00010000:1.0:1788845136.431290:0:13304:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.431297:0:13304:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.431305:0:13304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.431322:0:13304:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845136.431339:0:13304:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f03412680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2368:x1875739641886208:12345-192.168.201.29@tcp:103:ldlm_bl.0 Request processed in 98us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845136.431346:0:13304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 582 00010000:00010000:3.0:1788845136.431819:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1000/0xb2187301287af8f1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa06 expref: 13 pid: 5839 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845136.431837:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660451072:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845136.431851:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c0600/0xb2187301287af8d5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.29@tcp remote: 0x3100577d109aa9ff expref: 12 pid: 11036 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845136.431860:0:5840:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03592b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5840:1875739660451200:192.168.201.29@tcp:104: 00010000:00010000:3.0:1788845136.431912:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.431921:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.431927:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845136.431963:0:5840:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 0 00010000:00010000:3.0:1788845136.432276:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432285:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432291:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432297:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773200 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845136.432302:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432309:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432319:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a006406d00) 00010000:00010000:3.0:1788845136.432322:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788845136.432326:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 2/0,0 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432333:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af8ff lrc: 0/0,0 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432385:0:5840:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af8f8 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa0d expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432396:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845136.432398:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845136.432402:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af8f8 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa0d expref: 12 pid: 5840 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.432413:0:5840:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773a00, rc 0) 00000100:00100000:3.0:1788845136.432451:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f02383800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:69859fee-4322-4bed-b50b-7a298318b0f6+11:19024:x1875739641885952:12345-192.168.201.29@tcp:101:bash.0 Request processed in 32486us (32702us total) trans 4294967917 rc 301/301 00000100:00100000:3.0:1788845136.432458:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1738 00010000:00010000:2.0:1788845136.468631:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845136.482860:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845136.483254:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641886464 00000100:00100000:3.0:1788845136.483265:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641886464 00000100:00100000:3.0:1788845136.483424:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1739 00000100:00100000:3.0:1788845136.483435:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+11:19024:x1875739641886464:12345-192.168.201.29@tcp:36:bash.0 00010000:00010000:3.0:1788845136.483561:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.483588:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.483600:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845136.483735:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e4aa1a7d reply state 000000008f5441c5 transno 4294967918 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845136.483744:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845136.483835:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:19024:x1875739641886464:12345-192.168.201.29@tcp:36:bash.0 Request processed in 402us (986us total) trans 4294967918 rc 0/0 00000100:00100000:3.0:1788845136.483843:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1739 00000100:00100000:2.0:1788845136.515657:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845136.515767:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641886592 00000100:00100000:1.0:1788845136.515777:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641886592 00000100:00100000:1.0:1788845136.515836:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1740 00000100:00100000:1.0:1788845136.515847:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+13:19024:x1875739641886592:12345-192.168.201.29@tcp:101:bash.0 00010000:00010000:1.0:1788845136.515872:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788845136.515925:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af914 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aaa14 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.515993:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af914 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa14 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.516008:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f036c1c00/0xb2187301287af914 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa14 expref: 14 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.516031:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:1.0:1788845136.516034:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788845136.516038:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f036c1c00, rc 0) 00000100:00100000:1.0:1788845136.516101:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:19024:x1875739641886592:12345-192.168.201.29@tcp:101:bash.0 Request processed in 256us (454us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845136.516108:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1740 00010000:00010000:3.0:1788845136.523546:0:3298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.523575:0:3298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.523585:0:3298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.523592:0:3298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845136.523596:0:3298:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.523614:0:3298:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.523633:0:3298:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845136.523636:0:3298:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845136.523641:0:3298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.523647:0:3298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af90d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845136.532481:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845136.532854:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641886720 00000100:00100000:3.0:1788845136.532864:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641886720 00000100:00100000:3.0:1788845136.532958:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1741 00000100:00100000:3.0:1788845136.532969:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+13:19024:x1875739641886720:12345-192.168.201.29@tcp:101:bash.0 00010000:00010000:3.0:1788845136.532990:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845136.533010:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af91b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aaa1b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.533038:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af91b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000080000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa1b expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.533065:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773200/0xb2187301287af91b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000080000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa1b expref: 14 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845136.533094:0:5839:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:3.0:1788845136.533097:0:5839:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb mtime from disk: 0 -> 1788845136 00000004:00010000:3.0:1788845136.533099:0:5839:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb atime from disk: 0 -> 200000 00000004:00010000:3.0:1788845136.533102:0:5839:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb ctime from disk: 0 -> 1788845136 00000004:00010000:3.0:1788845136.533104:0:5839:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb blocks from disk: 18427282363563114435 -> 3 00010000:00010000:3.0:1788845136.533112:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845136.533114:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788845136.533118:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f773200, rc 0) 00000100:00100000:3.0:1788845136.533176:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:19024:x1875739641886720:12345-192.168.201.29@tcp:101:bash.0 Request processed in 209us (705us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845136.533183:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1741 00000100:00100000:3.0:1788845136.560528:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845136.560610:0:5852:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641886848 00000100:00100000:0.0:1788845136.560620:0:5852:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641886848 00002000:00010000:0.0:1788845136.560659:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0x13:0x0] 00002000:00100000:0.0:1788845136.560664:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03592b480 with x1875739641886848 00000100:00100000:0.0:1788845136.560703:0:5852:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1156 00000100:00100000:0.0:1788845136.560714:0:5852:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2370:x1875739641886848:12345-192.168.201.29@tcp:10:bash.0 00000004:00010000:0.0:1788845136.561024:0:5852:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:0.0:1788845136.561028:0:5852:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb mtime from disk: 1788845136 -> 1788845136 00000004:00010000:0.0:1788845136.561030:0:5852:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb atime from disk: 200000 -> 200000 00000004:00010000:0.0:1788845136.561033:0:5852:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb ctime from disk: 1788845136 -> 1788845136 00000004:00010000:0.0:1788845136.561035:0:5852:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb blocks from disk: 3 -> 3 00000100:00100000:0.0:1788845136.561110:0:5852:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:69859fee-4322-4bed-b50b-7a298318b0f6+15:2370:x1875739641886848:12345-192.168.201.29@tcp:10:bash.0 Request processed in 397us (593us total) trans 4294967919 rc 0/0 00000100:00100000:0.0:1788845136.561116:0:5852:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1156 00002000:00010000:0.0:1788845136.561120:0:5852:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0x13:0x0] 00002000:00100000:0.0:1788845136.561122:0:5852:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a03592b480 with x1875739641886848 00000100:00100000:1.0:1788845136.579318:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845136.582163:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641887104 00000100:00100000:3.0:1788845136.582180:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641887104 00000100:00100000:3.0:1788845136.582242:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 51 00000100:00100000:3.0:1788845136.582251:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+15:19024:x1875739641887104:12345-192.168.201.29@tcp:35:bash.0 00000100:00100000:3.0:1788845136.582519:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:69859fee-4322-4bed-b50b-7a298318b0f6+16:19024:x1875739641887104:12345-192.168.201.29@tcp:35:bash.0 Request processed in 267us (3212us total) trans 4294967920 rc 0/0 00000100:00100000:3.0:1788845136.582526:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 51 00000100:00100000:3.0:1788845136.597367:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:0.0:1788845136.597541:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641887232 00000100:00100000:0.0:1788845136.597550:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641887232 00000100:00100000:0.0:1788845136.597607:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1742 00000100:00100000:0.0:1788845136.597619:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:19167:x1875739641887232:12345-192.168.201.29@tcp:101:utime.0 00010000:00010000:0.0:1788845136.597643:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788845136.597666:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af922 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aaa22 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845136.597754:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af929 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845136.597770:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af929 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845136.597781:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af929 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788845136.597812:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788845136.597815:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788845136.597833:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af929 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845136.597855:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007800/0xb2187301287af922 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa22 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845136.597874:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af929 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa22 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788845136.597909:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929c100) 00010000:00010000:0.0:1788845136.597913:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788845136.597915:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff949f02007e00, rc 0) 00000100:00100000:0.0:1788845136.597982:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:19167:x1875739641887232:12345-192.168.201.29@tcp:101:utime.0 Request processed in 365us (623us total) trans 0 rc 0/0 00000100:00100000:0.0:1788845136.597988:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1742 00000100:00100000:0.0:1788845136.615605:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:1.0:1788845136.616267:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641887488 00000100:00100000:1.0:1788845136.616280:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641887488 00000100:00100000:1.0:1788845136.616344:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1743 00000100:00100000:1.0:1788845136.616353:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:19167:x1875739641887488:12345-192.168.201.29@tcp:36:utime.0 00010000:00010000:1.0:1788845136.616409:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788845136.616437:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff949f02007e00/0xb2187301287af929 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa22 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788845136.616455:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a01929c100) 00010000:00010000:1.0:1788845136.616467:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788845136.616827:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+16:19167:x1875739641887488:12345-192.168.201.29@tcp:36:utime.0 Request processed in 474us (1231us total) trans 0 rc -17/-17 00000100:00100000:1.0:1788845136.616834:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1743 00000100:00100000:0.0:1788845136.636012:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845136.642312:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641887616 00000100:00100000:2.0:1788845136.642328:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641887616 00000100:00100000:2.0:1788845136.642391:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1744 00000100:00100000:2.0:1788845136.642402:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+16:19167:x1875739641887616:12345-192.168.201.29@tcp:101:utime.0 00010000:00010000:2.0:1788845136.642425:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845136.642449:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af930 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aaa29 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.642528:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af937 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.642542:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af937 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.642553:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af937 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845136.642588:0:11036:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1788845136.642592:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845136.642612:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af937 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.642634:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864400/0xb2187301287af930 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa29 expref: 18 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.642653:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865000/0xb2187301287af937 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa29 expref: 17 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.642661:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6300) 00010000:00010000:2.0:1788845136.642664:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788845136.642668:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008865000, rc 0) 00000100:00100000:2.0:1788845136.644729:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+17:19167:x1875739641887616:12345-192.168.201.29@tcp:101:utime.0 Request processed in 2318us (8715us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845136.644742:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1744 00000100:00100000:2.0:1788845136.655751:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845136.655791:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641887744 00000100:00100000:2.0:1788845136.655801:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641887744 00000100:00100000:2.0:1788845136.655852:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1745 00000100:00100000:2.0:1788845136.655875:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03f655180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+17:19167:x1875739641887744:12345-192.168.201.29@tcp:101:utime.0 00010000:00010000:2.0:1788845136.655911:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788845136.655942:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864c00/0xb2187301287af93e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aaa30 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656007:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656020:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656032:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656044:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656051:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656091:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656359:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af953 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656369:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af953 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656377:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af953 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788845136.656411:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788845136.656431:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af953 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656445:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656450:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656457:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656462:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008865c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845136.656466:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656473:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656483:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6300) 00010000:00010000:2.0:1788845136.656486:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845136.656491:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656497:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008865c00/0xb2187301287af945 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656507:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656513:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656520:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656525:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a008864a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845136.656527:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656533:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656539:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff94a0087c6000) 00010000:00010000:2.0:1788845136.656541:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845136.656543:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656548:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864a00/0xb2187301287af94c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656592:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864c00/0xb2187301287af93e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa30 expref: 19 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656615:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af953 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa30 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.656622:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:2.0:1788845136.656624:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788845136.656627:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a008864e00, rc 0) 00000100:00100000:2.0:1788845136.656682:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03f655180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+18:19167:x1875739641887744:12345-192.168.201.29@tcp:101:utime.0 Request processed in 821us (940us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845136.656692:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1745 00000100:00100000:2.0:1788845136.667363:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845136.667562:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641888000 00000100:00100000:3.0:1788845136.667572:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641888000 00000100:00100000:3.0:1788845136.667632:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1746 00000100:00100000:3.0:1788845136.667643:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f01bb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+18:19167:x1875739641888000:12345-192.168.201.29@tcp:36:utime.0 00010000:00010000:3.0:1788845136.667701:0:11036:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788845136.667728:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a008864e00/0xb2187301287af953 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa30 expref: 18 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.667747:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845136.667751:0:11036:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788845136.667794:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.667808:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.667818:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788845136.668137:0:11036:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:3.0:1788845136.668141:0:11036:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb mtime from disk: 1788845136 -> 100000 00000004:00010000:3.0:1788845136.668144:0:11036:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb atime from disk: 200000 -> 200000 00000004:00010000:3.0:1788845136.668146:0:11036:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb ctime from disk: 1788845136 -> 1788845136 00000004:00010000:3.0:1788845136.668149:0:11036:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb blocks from disk: 3 -> 3 00000004:00010000:3.0:1788845136.668155:0:11036:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a5918cd7 reply state 00000000cc38e6a9 transno 4294967921 ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788845136.668164:0:11036:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788845136.668255:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f01bb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+19:19167:x1875739641888000:12345-192.168.201.29@tcp:36:utime.0 Request processed in 614us (912us total) trans 4294967921 rc 0/0 00000100:00100000:3.0:1788845136.668263:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1746 00000100:00100000:3.0:1788845136.668699:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845136.668735:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845136.668780:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641888128 00000100:00100000:2.0:1788845136.668786:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641888128 00002000:00100000:2.0:1788845136.668816:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0x13:0x0] req@ffff94a035929880 x1875739641888128/t0(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:242/0 lens 496/0 e 0 to 0 dl 1788845147 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845136.668825:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035929880 with x1875739641888128 00000100:00100000:2.0:1788845136.668846:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1157 00000100:00100000:2.0:1788845136.668852:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035929880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+19:2369:x1875739641888128:12345-192.168.201.29@tcp:4:bash.0 00000004:00010000:2.0:1788845136.669320:0:5854:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb size from disk: 0 -> 1 00000004:00010000:2.0:1788845136.669324:0:5854:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb mtime from disk: 100000 -> 1788845136 00000004:00010000:2.0:1788845136.669326:0:5854:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb atime from disk: 200000 -> 200000 00000004:00010000:2.0:1788845136.669328:0:5854:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb ctime from disk: 1788845136 -> 1788845136 00000004:00010000:2.0:1788845136.669330:0:5854:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb blocks from disk: 3 -> 3 00000100:00100000:2.0:1788845136.669373:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035929880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641888128:12345-192.168.201.29@tcp:4:bash.0 Request processed in 520us (674us total) trans 4294967922 rc 0/0 00000100:00100000:2.0:1788845136.669380:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1157 00002000:00100000:2.0:1788845136.669388:0:5854:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0x13:0x0] req@ffff94a035929880 x1875739641888128/t4294967922(0) o4->69859fee-4322-4bed-b50b-7a298318b0f6@192.168.201.29@tcp:242/0 lens 496/448 e 0 to 0 dl 1788845147 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788845136.669395:0:5854:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff94a035929880 with x1875739641888128 00000100:00100000:2.0:1788845136.669420:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641888256 00000100:00100000:2.0:1788845136.669424:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641888256 00000100:00100000:2.0:1788845136.669448:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1747 00000100:00100000:2.0:1788845136.669455:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a03592a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+21:2369:x1875739641888256:12345-192.168.201.29@tcp:16:kworker.0 00010000:00010000:2.0:1788845136.671158:0:3297:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.671173:0:3297:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.671181:0:3297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.671187:0:3297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f773a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:2.0:1788845136.671191:0:3297:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.671199:0:3297:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.671214:0:3297:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:2.0:1788845136.671217:0:3297:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788845136.671220:0:3297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788845136.671226:0:3297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f773a00/0xb2187301287af95a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788845136.677857:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845136.678238:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641888384 00000100:00100000:3.0:1788845136.678248:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641888384 00000100:00100000:3.0:1788845136.678310:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 1748 00000100:00100000:3.0:1788845136.678321:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+21:19167:x1875739641888384:12345-192.168.201.29@tcp:101:utime.0 00010000:00010000:3.0:1788845136.678346:0:11036:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788845136.678365:0:11036:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af961 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x3100577d109aaa37 expref: -99 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.678427:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.678442:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.678453:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.678468:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.678475:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.678482:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680592:0:11036:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af976 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680683:0:11036:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af976 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680698:0:11036:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af976 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788845136.680741:0:11036:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788845136.680761:0:11036:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af976 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680774:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680781:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680787:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680794:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772a00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845136.680798:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680805:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680816:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff94a0087c6300) 00010000:00010000:3.0:1788845136.680819:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845136.680823:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680829:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772a00/0xb2187301287af968 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680841:0:11036:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680848:0:11036:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.680854:0:11036:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.681037:0:11036:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff94a03f772c00 already unused, calling callback (ffffffffc21604f0) 00010000:00010000:3.0:1788845136.681043:0:11036:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.681050:0:11036:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.681068:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff94a00be8d200) 00010000:00010000:3.0:1788845136.681071:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788845136.681074:0:11036:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.681080:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772c00/0xb2187301287af96f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11036 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.681120:0:11036:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772e00/0xb2187301287af961 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa37 expref: 23 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.681135:0:11036:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff94a03f772200/0xb2187301287af976 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.29@tcp remote: 0x3100577d109aaa37 expref: 22 pid: 11036 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788845136.681143:0:11036:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff94a00be8da00) 00010000:00010000:3.0:1788845136.681146:0:11036:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788845136.681149:0:11036:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff94a03f772200, rc 0) 00000100:00100000:3.0:1788845136.681214:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+22:19167:x1875739641888384:12345-192.168.201.29@tcp:101:utime.0 Request processed in 2895us (3367us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845136.681224:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1748 00000001:00080000:2.0:1788845136.784097:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967919 is committed 00000001:00080000:2.0:1788845136.784124:0:5861:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967922 is committed 00000100:00100000:3.0:1788845136.784467:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a03592a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:69859fee-4322-4bed-b50b-7a298318b0f6+14:2369:x1875739641888256:12345-192.168.201.29@tcp:16:kworker.0 Request processed in 115007us (115725us total) trans 0 rc 0/0 00000100:00100000:3.0:1788845136.784480:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 1747 00010000:00010000:2.0:1788845137.493314:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788845138.133192:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01964700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660451456:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.133279:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845138.133338:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660451968:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.133364:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845138.133392:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788845138.133398:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f01d8fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:05b4ec57-2769-4797-9b3d-21582f14799e:3312:1875739660451328:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.133424:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845138.133443:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3312:1875739660451840:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.133462:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845138.133487:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660451584:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.133508:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845138.133527:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1788845138.133532:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a035b7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660451712:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.133713:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845138.133744:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660451456 00000100:00100000:1.0:1788845138.133752:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660451456 00000100:00100000:1.0:1788845138.133793:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660451840 00000100:00100000:1.0:1788845138.133795:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660451840 00000100:00100000:1.0:1788845138.133816:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660451584 00000100:00100000:1.0:1788845138.133818:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660451584 00000100:00100000:1.0:1788845138.133833:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1788845138.133853:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660451456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.133994:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1875739660451456:12345-0@lo:400:kworker.0 Request processed in 141us (715us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845138.134001:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1788845138.134019:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:1.0:1788845138.134024:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1875739660451840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.134063:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1875739660451840:12345-0@lo:400:kworker.0 Request processed in 39us (599us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845138.134070:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00100000:1.0:1788845138.134081:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:1.0:1788845138.134087:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660451584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.134117:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3313:x1875739660451584:12345-0@lo:400:kworker.0 Request processed in 30us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845138.134123:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:1.0:1788845138.134159:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3313:1875739660451584:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.134183:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3312:1875739660451840:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.134193:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01964700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3312:1875739660451456:0@lo:400:kworker.0 00000100:00100000:2.0:1788845138.134660:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660451328 00000100:00100000:2.0:1788845138.134665:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660451328 00000100:00100000:2.0:1788845138.134703:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1788845138.134711:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3312:x1875739660451328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845138.134756:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:05b4ec57-2769-4797-9b3d-21582f14799e+19:3312:x1875739660451328:12345-0@lo:400:kworker.0 Request processed in 46us (1331us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845138.134762:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1788845138.135400:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660451968 00000100:00100000:2.0:1788845138.135406:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660451968 00000100:00100000:2.0:1788845138.135438:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660451712 00000100:00100000:2.0:1788845138.135441:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660451712 00000100:00100000:2.0:1788845138.135463:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:2.0:1788845138.135470:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660451968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845138.135507:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3312:x1875739660451968:12345-0@lo:400:kworker.0 Request processed in 38us (2142us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845138.135513:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:2.0:1788845138.135524:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1788845138.135530:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff949f033f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660451712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788845138.135582:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff949f033f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660451712:12345-0@lo:400:kworker.0 Request processed in 52us (1869us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845138.135588:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1788845138.135623:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660451712:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.136388:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f01d8fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:05b4ec57-2769-4797-9b3d-21582f14799e:3312:1875739660451328:0@lo:400:kworker.0 00000100:00100000:1.0:1788845138.136403:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a035b7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3312:1875739660451968:0@lo:400:kworker.0 00000100:00080000:1.0:1788845138.137043:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1788845138.137047:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788845138.137050:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1788845138.514460:0:5828:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788845138.642185:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788845138.642213:0:3314:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff94a03f775880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660452096:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788845138.642288:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788845138.642392:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660452096 00000100:00100000:2.0:1788845138.642400:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660452096 00000100:00100000:2.0:1788845138.642457:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:2.0:1788845138.642467:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035684000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660452096:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788845138.642501:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95354880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788845138.642551:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035684000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1875739660452096:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (263us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845138.642561:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00100000:2.0:1788845138.642604:0:3314:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff94a03f775880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3314:1875739660452096:0@lo:13:osp-pre-1-0.0 00000001:02000400:0.0:1788845138.693711:0:16140:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_36b: @@@@@@ FAIL: utime failed for OST 00010000:00010000:2.0:1788845139.539156:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788845139.668953:0:3313:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788845139.668983:0:3313:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff949f033f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660452224:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845139.669091:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788845139.669242:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739660452224 00000100:00100000:1.0:1788845139.669250:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739660452224 00000100:00100000:1.0:1788845139.669308:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:1.0:1788845139.669318:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660452224:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788845139.669357:0:6686:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788845139.669422:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3313:x1875739660452224:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845139.669429:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:1.0:1788845139.669474:0:3313:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff949f033f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3313:1875739660452224:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788845139.738277:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845139.738309:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845139.738339:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:3.0:1788845139.738366:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.29@tcp (source: 12345-192.168.201.29@tcp) 00000100:00100000:2.0:1788845139.739457:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641888768 00000100:00100000:2.0:1788845139.739470:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641888768 00000100:00100000:2.0:1788845139.739541:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641888896 00000100:00100000:2.0:1788845139.739545:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641888896 00000100:00100000:2.0:1788845139.739570:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 189 00000100:00080000:3.0:1788845139.739578:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788845139.739579:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641888768:12345-192.168.201.29@tcp:400:kworker.0 00000100:00080000:3.0:1788845139.739583:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788845139.739659:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+5:2367:x1875739641888768:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 81us (1348us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845139.739667:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 189 00000100:00100000:2.0:1788845139.739684:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 190 00000100:00100000:2.0:1788845139.739690:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641888896:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:2.0:1788845139.739714:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:69859fee-4322-4bed-b50b-7a298318b0f6+6:2367:x1875739641888896:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 24us (1348us total) trans 0 rc 0/0 00000100:00100000:2.0:1788845139.739720:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 190 00000100:00100000:1.0:1788845139.739798:0:5833:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641888512 00000100:00100000:1.0:1788845139.739803:0:5833:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641888512 00000100:00100000:1.0:1788845139.739842:0:5833:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 333 00000100:00100000:1.0:1788845139.739850:0:5833:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a00915d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2367:x1875739641888512:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845139.739878:0:5833:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a00915d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1b3b756c-2345-49bb-b1ef-d6c30ed3e37c+10:2367:x1875739641888512:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 29us (1612us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845139.739912:0:5833:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 333 00000100:00100000:1.0:1788845139.739941:0:11036:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875739641888640 00000100:00100000:1.0:1788845139.739945:0:11036:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875739641888640 00000100:00100000:1.0:1788845139.739981:0:11036:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.29@tcp, seq: 255 00000100:00100000:1.0:1788845139.739987:0:11036:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff94a035b7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641888640:12345-192.168.201.29@tcp:400:kworker.0 00000100:00100000:1.0:1788845139.740029:0:11036:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff94a035b7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:69859fee-4322-4bed-b50b-7a298318b0f6+14:2367:x1875739641888640:12345-192.168.201.29@tcp:400:kworker.0 Request processed in 42us (1690us total) trans 0 rc 0/0 00000100:00100000:1.0:1788845139.740036:0:11036:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.29@tcp, seq: 255 00000100:00080000:1.0:1788845139.740058:0:5835:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 Debug log: 71617 lines, 71617 kept, 0 dropped, 0 bad.